3d+ imaging systems with on-chip neighbor-in-time analog sub-pixel processing and single-pixel motion determination

ABSTRACT

Various embodiments of a 3D+ imaging system include imaging systems with on-chip neighbor-in-time analog sub-pixel processing and single-pixel motion determination. In embodiments, range gating imaging techniques are used to generate a composite image depth map of a scene based on analog sub-pixel processing are implemented. In embodiments, high-intensity rate of change of neighboring pixels techniques are used to generate a composite image two-axis motion map of a scene at a pixel level. In embodiments, an analog pixel circuit is disclosed for use with an array of photodetectors for a sub-frame composite imaging system. In embodiments, an extended-dynamic-range imaging technique is used in imaging to reproduce a greater dynamic range of luminosity.

FIELD OF THE INVENTION

The present disclosure relates to camera and imaging systems. Moreparticularly, the present disclosure relates imaging systems thatgenerate three-dimensional and higher-dimensional composite images withfaster and more expansive per pixel data generated by on-chip processingthat can include neighbor-in-time analog sub-pixel processing and/orsingle-pixel motion determination.

BACKGROUND OF THE INVENTION

Three-dimensional (3D) cameras, four-dimensional (4D) cameras, andrelated high performance imaging systems, referred to in this disclosureas 3D+ imaging systems, are capable of providing more than justtwo-dimensional images of a scene. 3D+ imaging systems can provide, forexample, distance measurements, motion measurements, and/or photonicmeasurements for physical objects in a scene. An example of a 3D+ camerasystem that generates lighting-invariant images is disclosed in U.S.Pat. No. 10,382,742.

One of the earliest on-chip image processing systems was the SCAMP chip.(https://www.semanticscholar.org/paper/A-general-purpose-CMOS-vision-chip-with-a-SIMD-Dudek-Hicks/9562f3b610a912ba4ccac1ae463aad87638b4dc1).The most current version of SCAMP chip is the SCAMP-5 chip whichfeatures a high speed analog VLSI image acquisition and low-level imageprocessing system. The architecture of the SCAMP-5 chip is based on adynamically reconfigurable SIMD processor array that features amassively parallel architecture enabling the computation of programmablemask-based image processing in each pixel.(https://personalpages.manchester.ac.uk/staff/p.dudek/scamp/). The chipcan capture raw images up to 10,000 fps and runs low-level imageprocessing at a frame rate of 2,000-5,000 fps.

Various examples of on-chip processing systems for high performanceimaging systems are described U.S. Pat. Nos. 8,102,426, 8,629.387,9,094,628, and 10,218,913, U.S. Publ. Appls. US 2019/0033448A1, US2019/0056498A1, and(https://ieeexplore.ieee.org/abstract/document/7527519).

SUMMARY OF THE INVENTION

An imaging system configured to generate a composite image depth map ofa scene in accordance with various embodiments comprises at least oneemitter configured to emit an active light pulse toward the scene and anarray of detectors configured to receive light that includes some of theactive light pulse reflected from the scene for a field of view thatincludes at least a portion of the scene. Control circuitry is operablycoupled to the at least one emitter and the array of detectors and to aprocessing system. The control circuitry is configured to cause the atleast one emitter to emit the active light pulse and to cause the arrayof detectors to receive light to store at least three successivesub-frames of analog data as a sub-frame pixel in one or more sub-framebuffers, wherein each sub-frame pixel has a timing relationship of anemitter/detector cycle for that sub-frame pixel. The processing systemconfigured to analyze the at least three successive sub-frames pixels todetermine for a pixel associated with the sub-frame pixels a blackpoint, a white point, and the sub-frame pixel at which the white pointoccurs, and determine a distance range for each pixel based on thesub-frame pixel at which the white point occurs.

In embodiments, the distance range represented by each sub-frame pixelis defined by an overlap in a duration of the timing relationship of theemitter/detector cycle for that sub-frame pixel. In embodiments, a totaldistance range of the imaging system is equal to a number of sub-framepixels per pixel times the distance range of each sub-frame pixel.

In some embodiments, the imaging system is mounted in a vehicle capableof moving at speeds of more than 50 km/hour and all of the three or moresub-frame pixels for each pixel are stored within an imaging window lessthan 250 μSec. In some embodiments, the imaging system is mounted in ahandheld device and the three or more sub-frame pixels for each pixelare stored within an imaging window of less than 2500 μSec.

An imaging system configured to generate a composite image two-axismotion map of a scene at a pixel level in accordance with variousembodiments comprises at least one emitter configured to emit an activelight pulse toward the scene and an array of detectors configured toreceive light that includes some of the active light pulse reflectedfrom the scene for a field of view that includes at least a portion ofthe scene. Control circuitry is operably coupled to the at least oneemitter and the array of detectors and to a processing system. Thecontrol circuitry is configured to store a set of at least threesuccessive sub-frames of intensity data as sub-frame pixels in one ormore sub-frame buffers, wherein each sub-frame pixel has a timingrelationship of an emitter/detector cycle for that sub-frame pixel andeach set of sub-frame pixels associated with a unique one of an array ofpixels based on a row and a column corresponding to the array ofdetectors. The processing system is configured to analyze at least threesuccessive sub-frame pixels to determine for each pixel in the array ofpixels a black point due to ambient light in the scene and a white pointdue to the active light pulse reflected from the scene for at least thefirst sub-frame pixel and the last sub-frame pixel for the set ofsub-frame pixels for that pixel, generate a horizontal axis motion valuefor each pixel relative to a row in the pixel array based on ahigh-intensity rate of change between that pixel and at least oneneighbor pixel in the row; and generate a vertical axis motion value foreach pixel relative to a column in the pixel array based on ahigh-intensity rate of change between that pixel and at least oneneighbor pixel in the column.

In embodiments, the processing system determines the high-intensity rateof change by evaluating a sub-frame pixel at which a trapezoid slope ofthe white point crosses over a trailing edge of the black point for aneighbor pixel in a pixel triplet for a given row or column of the pixelarray that has a trapezoid slope that is non-zero.

In some embodiments, a duration of a capture cycle is constant for theat least three successive sub-frame pixels, and an intensity and aduration of the active light pulse emitted during the capture cycle isthe same for the first sub-frame pixel and the last sub-frame pixel, butthe intensity and the duration of the active light pulse is differentfor at least one sub-frame pixel between the first sub-frame pixel andthe last sub-frame pixel. In some embodiments, a duration of a captureis the same for the first sub-frame pixel and the last sub-frame pixel,but the duration of a capture is shorter for at least one sub-framepixel between the first sub-frame pixel and the last sub-frame pixel.

In some embodiment, the array of detectors, the control circuitry andthe processing system are integrated on a single chip. In someembodiments, the array of detectors and the control circuitry areintegrated on a single chip and the processing system is external to thesingle chip.

In embodiments, the active light pulse in a given emitter/detector cyclefor a given pixel comprises a number of pulses selected from the setconsisting of a single pulse per pixel, a sequence of multiple pulsesper pixel, a single pulse per sub-pixel, or multiple pulses persub-pixel, and a frequency selected from the set consisting of a singlefrequency range or multiple frequency ranges.

In some embodiments, the array of detectors is configured to accumulatelight based on a single accumulation for the timing relationship of theemitter/detector cycle that is unique for each sub-pixel.

In embodiments, an on-chip image processing system for 3D+ imagingutilizes sub-frame, composite image, and/or trapezoid analysis.

In embodiments, an on-chip image processing system for 3D+ imagingprovides for two-axis motion determination for white point/black pointcomposite imagery and three-axis motion determination for trapezoidalcomposite imagery, both motion determinations being based on a single,composite image instead of motion analysis done over multiple images.

In embodiments, an on-chip image processing system for 3D+ imagingincreases the images-per-second processing rate with decreased powerconsumption.

In embodiments, an on-chip image processing system for 3D+ imaginggenerates a range gating depth map method whereby the depth calculationis independent of the reflectivity characteristic of the objects in thescene.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an electrical circuit for a prior art single-frameimaging pixel for a five-transistor (5T) configuration.

FIG. 1B illustrates a prior art switched-current memory.

FIG. 2 illustrates an electrical circuit for a sub-frame imaging pixelin accordance with an embodiment utilizing a 5T architecture.

FIG. 3 illustrates a functional implementation of an embodiment for ananalog shift register using SI memory.

FIG. 4 illustrates emitter and detector timing for an embodiment withthirty-two sub-frames used to produce sub-frame composite images.

FIG. 5 illustrates a plot of intensity values vs. sub-frame number forthe thirty-two-sub-frame imaging cycle shown in FIG. 4 .

FIG. 6 illustrates a vehicle traversing a roadway with asub-frame-processing, composite-imaging camera in accordance with anembodiment.

FIG. 7 illustrates a waveform produced with timing parameters accordingto an embodiment with a trapezoid descriptor for a 32/8/8/12/1/5configuration

FIG. 8 illustrates components of an embodiment of asub-frame-processing, composite imaging system.

FIG. 9 illustrates a block diagram of a prior art SCAMP-5 FSPS.

FIG. 10 illustrates a functional diagram for an embodiment of a pixel ina neighbor-in-time analog pixel processing (NitAPP) configuration.

FIG. 11 illustrates a detailed block diagram of an embodiment of aNitAPP circuitry showing analog storage and processing elementsconnected to an analog bus.

FIG. 12 illustrates an S²I description of register circuitry accordingto an embodiment.

FIG. 13 illustrates an analog multiplier functional block and a S²Idescription of a multiplier according to an embodiment.

FIG. 14 illustrates a compare-and-flag functional block and a S²Idescription of compare and flag circuitry according to an embodiment.

FIG. 15 illustrates a current source functional block and a SIdescription of enabling a known current source onto an analog busaccording to an embodiment.

FIG. 16 illustrates components of embodiments for asub-frame-processing, composite imaging system that utilizes NitAPPprocessing.

FIG. 17 illustrates a scene that is imaged by a sub-frame compositeimage camera in accordance with an embodiment.

FIG. 18 illustrates the optical timing parameters for an embodiment of anon-overlapping range gating descriptor of 3/100/1/2/0.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 illustrates an electrical circuit 10 for a single-frame prior artimaging pixel for a five-transistor (5T) configuration. The sevencomponents consist of a photodetector 12, a storage capacitor 14, aphotodetector reset switch 16, a photodetector charge transfer switch18, a capacitor reset switch 20, a non-destructive read transistor 22,and a selector switch 24. A photodetector 12 will convert optical energyto electrical energy and will store a charge at the photodetector whenthe photodetector reset switch 16 and the photodetector charge transferswitch 18 are both open. The amount of charge stored at thephotodetector will depend on the number of incident photons during theintegration period and the quantum efficiency of the photodetectormaterial. At the end of an integration event, the charge collected atthe photodetector 12 is transferred to the storage capacitor 14 byclosing the transfer switch 18. Upon charge transfer, the integratedphotodetector value is transferred from the pixel by closing theselector switch 24. The 5T pixel, as shown, supports multipleaccumulations whereby multiple photodetector accumulation cycles areperformed prior to closing the selector switch 24, the closing of whichcompletes a sub-frame photodetector event. Imaging circuitry thatsupports multiple accumulation capability allows imaging systems toreduce noise, enables a higher signal-to-noise ratio (SNR), allows forshorter individual integration times when utilizing neighbor-in-timecomposite imaging and when utilizing range gating, and allows forincreasing the amount of emitter light that is integrated duringphotodetector sub-frame cycles.

FIG. 2 illustrates an electrical circuit for a sub-frame imaging pixel40 in accordance with an embodiment utilizing a 5T architecture. Inembodiments, the sub-frame circuitry has three elements—a photodetectorcircuitry 42, analog shift register circuitry 46, and output circuitry44. A value for one or more sub-frame integrations that is stored at thepixel capacitor 45 is provided to the input of an analog shift register46. The analog value at the capacitor 45 is shifted to bit K−1 of theshift register 46 when the Shift 48 signal is activated. All analogvalues previously stored in the shift register 46 are shifted to thesubsequent bit when Shift 48 is activated, and the pre-shifted valuecontained in bit 0 is lost. The select switch 44 enables bit 0 onto thepixel output bus. In embodiments, the number of bits in an analog shiftregister corresponds to the number of sub-frames for a composite image.

As shown in FIG. 1B, a prior art switched-current memory (referred toherein as SI memory) is a memory that stores an analog value representedby a current in a capacitor and uses the I_(D)−V_(GS) relation of a MOStransistor to do the conversion between current and voltage. With adiode-connected transistor collecting a current I_(in), the transistordevelops a gate-source voltage according to I_(in)=β(V_(GS)−V_(TH))².When the connection between gate and drain is broken, the charge in thegate has nowhere to go and remains constant. When the transistor isworking as a current source, it produces the same current as itcollected. The MOS transistor converts the current into voltage andstores it. Afterwards, it converts the gate voltage into the samecurrent and supplies it.

SI memory has limitations for data accuracy and current draw. It is,however, an effective way to show functional current-switched logic.FIG. 3 illustrates a functional implementation of an embodiment for ananalog shift register using SI memory. In embodiments, the shifting ofan analog memory value from register k to k−1 involves two stages—atransfer stage that receives the value from the previous register orinput, and a shift stage that stores the value in the register outputcell. A K-bit analog shift register 50 is shown as a functional symboland in an expanded view, whereby the expanded view illustrates SI logicused for a bit K−1 register 52 and a bit 0 register 54. Intermediatebits of the K-bit analog shift register are represented by continuationdots, with signal and data interconnects represented with dashed lines.When the XFER 58 signal is active the transfer data switch 68 enablesbit K−1 input 56 current to the source of the transfer storagetransistor 62. When the XFER 58 signal is active the transfer storageswitch 66 enables current flow to the gate of the transfer storagetransistor 62. Upon deactivation of the XFER 58 signal, the originalinput data value is stored at the transfer storage transistor 62. Whenthe Shift 60 signal is active the shift data switch 72 enables the datainput current to the source of the shift storage transistor 64. When theShift 60 signal is active the shift storage switch 70 enables currentflow to the gate of the shift storage transistor 64. Upon deactivationof the Shift 60 signal, the transfer data value is stored at the shiftstorage transistor 64. For the bit 0 register 54, the source of theshift storage transistor provides the bit 0 output 74 of the K-bitanalog shift register.

In embodiments, sub-frame capture and processing produces compositeimages. An example of composite imagery created with sub-frames isdisclosed in U.S. Pat. No. 9,866,816 (Retterath) for an Active Pulsed 4DCamera, and this patent is incorporated by reference herein. FIG. 4illustrates emitter and detector timing for thirty-two sub-frames usedto produce sub-frame composite images. Although the sub-frame timing isshown on the same time scale, sub-frame 0 through sub-frame 31 captureoccurs at disparate and consecutive times within a thirty-two sub-framecapture window. Emitter pulse sub-frame 0 80 is shown using an activehigh “on” signal and integration sub-frame 0 82 is shown using an activehigh “on” signal. In embodiments, there exists no time overlap betweenemitter pulse sub-frame 0 and integration sub-frame 0. The duration ofthe time difference between the end of integration sub-frame 0 82 andthe start of emitter sub-frame 0 80 is denoted by the sub-frame 0 offsetintegration-end-to-emitter-start 88. In embodiments, the timing ofintegration and emitter signals is referenced relative to an emitterclock period duration 85. In embodiments, an emitter pulse duration 86and an integration duration 84 are produced as multiple increments of anemitter clock period duration 85.

In accordance with various embodiments described herein sub-framecapture may utilize varying photodetector integration times forsub-frames within a passive composite image and will utilize varyingtiming relationships between emitter active and photodetectorintegration times. In some embodiments, the sub-frame processingtechniques rely on the use of photodetector responses that arelinearized. For active camera system embodiments, multiple emitterwavelengths may be utilized for the various modes. Multiple wavelengthsmay be emitted during a sub-frame cycle, or single wavelengths may beemitted during a single sub-frame cycle with a different wavelengthbeing emitted during subsequent sub-frames within a composite imagingwindow. Not all operational modes of various embodiments utilizelinearization; however, for operational modes that utilize photodetectorlinearization, photodetectors that respond to multiple wavelengths musthave a linearization capability for every wavelength modality of theemitter(s). As an example, for a photodetector array that contains aBayer filter, the individual photodetectors may have a green, blue orred filter associated with the photodetector and will have differingresponses to visible light (400-700 nm) and to narrowband NIR light like850 nm. Photodetectors with a red filter that are used in multispectralcomposite image sub-frame processing in accordance with this embodimentwould require a linearization function for visible light and alinearization function for NIR light. Photodetectors with a green filterthat are used in multispectral composite image sub-frame processing inaccordance with this embodiment would require a linearization functionfor visible light and a linearization function for NIR light.Photodetectors with a blue filter that are used in multispectralcomposite image sub-frame processing in accordance with this embodimentwould require a linearization function for visible light and alinearization function for NIR light.

Sub-frame integration for a sub-frame composite imaging cycle willresult in an intensity value for each pixel (m,n) in an imaging array.FIG. 5 illustrates a plot 90 of intensity values vs. sub-frame numberfor a thirty-two-sub-frame imaging cycle. In embodiments, based on theemitter and integration times for the sub-frames whereby there is noemitter-detector overlap at sub-frame 0 and there is no emitter-detectoroverlap at sub-frame 31, the resulting intensity waveform 90 is atrapezoid for all pixels that image objects within a camera's imagingrange. Pixels outside the camera's range and imaging in attenuatingenvironments will typically result in waveforms that contain partialtrapezoidal elements. For an isosceles trapezoidal waveform 90, thevalue of the base of the trapezoid represents a black point 92 intensityand the value of the top of the trapezoid represents a white point 92intensity. Four trapezoid inflection points 96, 98, 100, 102 are definedas the leading-edge-black-inflection-point 96 [IP0(i,sf)], theleading-edge-white-inflection-point 98 [IP1(i,sf)],trailing-edge-white-inflection-point 100 [IP2(i,sf)], andtrailing-edge-black-inflection-point 102 [IP3(i,sf)]. The center of massof the trapezoid based on white point inflection points is definedaccording to:

COM _(WhitePoint)(sf)=(IP2(sf)−IP1(sf))/2  Eq. 1

Where IP2(sf) is the sub-frame value of IP2(i,sf)

-   -   IP1(sf) is the sub-frame value of IP1(i,sf)

Alternatively, the center of mass of the trapezoid based on black pointinflection points is defined according to:

CoM _(Blackpoint)(sf)=(IP3(sf)−IP0(sf))/2  Eq. 2

Where IP3(sf) is the sub-frame value of IP3(i,sf)

-   -   IP0(sf) is the sub-frame value of IP0(i,sf)

For an isosceles trapezoidal, sub-frame composite image pixel waveform,Eqs. 1 and 2 yield equivalent results. In embodiments, sub-framecomposite image timing is implemented with thirty-two sub-frames, anemitter clock period of 8 nanoseconds, an integration time of twelveemitter clock periods, an emitter pulse width of eight emitter clockperiods, a sub-frame 0 offset from detector end to emitter start of oneemitter clock period, and a sub-frame period duration of 5 82 Sec. Basedon these parameters, the shape, size, and horizontal location ofisosceles trapezoidal waveforms is defined sufficiently to allow atrapezoidal descriptor to enable the computation of a distance parameterfor every pixel in an array. A sub-frame trapezoid descriptor for a32/8/8/12/1/5 configuration is shown:

Sub-frame trapezoidal descriptor parameters:

# of sub-frames 32 Emitter clock period 8 nSec Emitter pulse width 8emitter clock periods Integration width 12 emitter clock periodsSub-frame 0 integration 1 emitter clock period end to emitter startSub-frame period duration 5 μSec Speed of light constant 0.299792 m/nSec(in a vacuum)

Sub-frame trapezoidal descriptor derived values:

Trapezoid IP0(sf) at d = 0 1 Trapezoid IP1(sf) at d = 0 9 TrapezoidIP2(sf) at d = 0 13 Trapezoid IP3(sf) at d = 0 21 Trapezoid CoM(sf) at d= 0 11 Trapezoid lower base width 20 Trapezoid upper base width 4Trapezoid width at mid-height 12 Range of camera 24.0 meters

Trapezoidal descriptor parameters are used to identify other trapezoidparameters and are used to identify inflection point “locations” and aCoM “location” at a distance of d=0, where location refers to thesub-frame number at which the point intersects the horizontal axis of anisosceles trapezoid pixel plot. In embodiments, sub-frame locations forpoints are specified in floating point values, thus yielding higheraccuracy for pixel distance determinations. According to the trapezoidaldescriptor derived values, the four inflection points at d=0 are atsub-frames 1, 9, 13, and 21 for the four inflection points IP0(sf),IP1(sf), IP2(sf), IP3(sf), respectively. The trapezoid CoM(sf) at d=0 isat sub-frame 11 and is computed by using Eq. 1 or Eq. 2. For compositeimage post-processing, the distance for each pixel is determined bycomputing the delta between the CoM(sf) value for pixel (m,n) and theCoM(sf) for d=0, where:

ΔCoM _([m,n]) =CoM _((m,n))(sf)—CoM _(d=0)(sf)  Eq. 3

Where CoM_((m,n))(sf) is the CoM of a trapezoid for pixel [m,n]

-   -   CoM_(d=0)(sf) is the CoM at d=0 from the trapezoid descriptor

The distance for pixel (m,n), where distance is defined as the measurefrom the camera to the object represented by pixel (m,n), is computedaccording to:

Distance_([m,n])=(ΔCoM _((m,n)) *C P _(emitter))/2  Eq. 4

Where C is a constant for the speed of light in a medium

-   -   P_(emitter) is the emitter clock period

In embodiments, the range of a sub-frame composite imaging camera may bespecified in various ways, depending on the shape and structure of theresulting waveform. For isosceles trapezoidal waveforms, the range isdefined as the maximum pixel distance for which an isosceles trapezoidalwaveform lies completely within the sub-frame range for a trapezoidaldescriptor. Said another way, the maximum range of a distance-measuringcamera that utilizes sub-frame collection and isosceles trapezoidalwaveform processing is defined as the point at which IP3(sf) is equal tothe maximum sub-frame number. In embodiments, the center of mass for amaximum range isosceles trapezoid is computed according to:

COM _(MaxRange) =SF _(max)−(Width_(LowerBase)/2)  Eq. 5

Where SF_(max) is the maximum sub-frame number

-   -   Width_(LowerBase) is the width of the lower trapezoid base

In embodiments, the maximum device range for distance measurements iscomputed according to:

Range_(max) ={[SF _(max)−(Width_(LowerBase)/2)−CoM _(d=0)(sf)]*C*P_(emitter)}/2  Eq. 6

Where SF_(max) is the maximum sub-frame number

-   -   Width_(LowerBase) is the width of the lower trapezoid base    -   CoM_(d=0)(sf) is the sub-frame for the CoM at which d=0    -   C is a constant for the speed of light in a medium    -   P_(emitter) is the emitter clock period

FIG. 6 illustrates a vehicle 110 traversing a roadway with the vehicle110 including a sub-frame-processing-composite-imaging camera 112. Afield-of-view (FOV) 114 of an unprocessed collection of sub-frames ofthe camera 112 is illustrated whereby sub-frames include collectedinformation for a near pavement marking 116, an intermediate-distanceroad sign 118, and a farther-distance roadway point 120. The nearpavement marking intensity plot indicates a black point intensity 122, awhite point intensity 124, and a CoM 126. The intermediate signintensity plot indicates a black point intensity 128, a white pointintensity 130, and a CoM 132. The far pavement surface intensity plotindicates a black point intensity 134, a white point intensity 136, anda CoM 138. The FOV 114 description indicates that sub-frame collectionfor all objects within a camera's range occurs within amultiple-sub-frame composite imaging window.

The use of inflection points for center of mass calculations forisosceles trapezoids leads to decreased precision for distancecalculations when inflection points do not correspond to integer valuesof sub-frame numbers. In embodiments, this limitation is removed byutilizing waveform mid-height crossover points to determine center ofmass. FIG. 7 illustrates a waveform 140 produced with timing parametersaccording to a trapezoid descriptor for a 32/8/8/12/1/5 configuration. Ablack point intensity level 142 is determined from the minimum intensityfor the K sub-frame intensity values, and a white point intensity level144 is determined from the maximum intensity for the K sub-frameintensity values. In embodiments, a mid-height intensity 146 value isthe average of the black point and white point value and is definedaccording to:

I _(mid-ht)(m,n)=(I _(BlackPoint)(m,n)+I _(WhitePoint)(m,n))/2  Eq. 7

The slope of the leading edge of the waveform is computed according to:

Slope_(LeadingEdge)=(I _(BlackPoint)(m,n)−I_(WhitePoint)(M,n))/(IP1(sf)−IP0(sf))  Eq. 8

The slope of the trailing edge of the waveform is computed according to:

Slope_(TrailingEdge)=(I _(BlackPoint)(m,n)−I_(WhitePoint)(m,n))/(IP3(sf)−IP2(sf))  Eq. 9

In embodiments, an algorithm for determining the mid-height crossoverpoints 148, 150 for the leading edge 148 and trailing edge 150 consistsof a process of incrementing sub-frame numbers and identifying thesub-frame number at which the leading and trailing edge waveforms crossover the computed mid-height intensity 146 value. The leading-edgeremainder 152 is the intensity value difference between the leading edgesub-frame crossover point intensity value and the mid-height intensity146 value. The sub-frame value at which the leading edge crosses overthe mid-height intensity is computed according to Eq. 10 below:

SF _(mid-ht-lead)(m,n)=SF _(mid-ht-exc-lead)(m,n)−[(I_(mid-ht-exc-lead)(m,n)−I _(mid-ht)(m,n))/Slope_(LeadingEdge)]

-   -   Where SF_(mid-ht-exc-lead)(m,n) is the leading-edge sub-frame at        which the intensity exceeds the mid-height value for pixel (m,n)        -   I_(mid-ht-exc-lead)(m,n) is the intensity value for the            leading-edge sub-frame at which the intensity exceeds the            mid-height value for pixel (m,n)        -   Slope_(LeadingEdge) is the slope of the leading edge of the            trapezoid

The trailing-edge remainder 154 is the intensity value differencebetween the trailing edge sub-frame crossover point intensity value andthe mid-height intensity 146 value. The sub-frame value at which thetrailing edge crosses over the mid-height intensity is computedaccording to Eq. 11 below:

SF _(mid-ht-trail)(m,n)=SF _(mid-ht-exc-trail)(m,n)−[(i_(mid-ht-exc-trad)(m,n)−I _(mid-ht)(m,n))/Slope_(TrailingEdge)]

-   -   Where SF_(mid-ht-exc-trail)(m,n) is the trailing-edge sub-frame        at which the intensity exceeds the mid-height value for pixel        (m,n)        -   i_(mid-ht-exc-trail)(m,n) is the intensity value for the            trailing-edge sub-frame at which the intensity exceeds the            mid-height value for pixel (m,n)        -   Slope_(TrailingEdge) is the slope of the trailing edge of            the trapezoid

The CoM 156 of the waveform 140 is the mid-point of the leading-edgecrossover point 148 and the trailing-edge crossover point 150 and iscomputed according to:

CoM(m,n)=(SF _(mid-ht-lead)(m,n)+SF _(mid-ht-trail)(m,n))/2  Eq. 12

The computed trapezoid mid-height width 158 is the offset (insub-frames) between of the leading-edge crossover point 148 and thetrailing-edge crossover point 150 and is computed according to:

Width_(MidHeight)(m,n)=sf _(mid-ht-trail)(m,n)−sf_(mid-ht-lead)(m,n)  Eq. 13

For an isosceles trapezoid, the computed value of the mid-height width158 will be equivalent to the mid-height width from the trapezoidaldescriptor. Variations between the computed mid-height width 158 and thecorresponding trapezoidal descriptor value are indications of scenarioslike imaging in attenuating environments or motion of objects in a sceneand/or motion of a camera.

In embodiments, an algorithm is specified for execution on a one or moreCPUs or GPUs for determining black point, white point and CoM for eachpixel (m,n) in a sub-frame, composite imaging system and is as follows:

CPU/GPU Pseudocode Constants: SF = number of sub-frames per compositeimage M = number of columns of pixels in FPA N = number of rows ofpixels in FPA CPU/GPU instruction CPU/GPU Comment Begin m = 0 initializecolumn counter n = 0 initialize row counter LoopMN k = 0 initialize loopcount for BP, WP BlackPoint[m,n] = 0x3FF initialize BP to a high valueWhitePoint[n,m] = 0 initialize WP to a low value LoopWP Read i[m,n,k]read bit k from shift memory If i[m,n,k] < BlackPoint[m,n] bit k lowestso far?  BlackPoint[m,n] = i[m,n,k] if yes, make bit sf new lowest endifIf i[m,n,k] > WhitePoint[m,n] bit k highest so far?  WhitePoint[m,n] =i[m,n,k] if yes, make bit k new highest endif k = k+1 incrementsub-frame counter If k<K, GoTo LoopWP end of MidHeight[m,n] =WhitePoint[m,n] − BlackPoint[m,n] mid-height intensity valueLeadEdgeMidPassed[m,n] = FALSE initialize leading edge CoM flag k = 0initialize loop count for CoM TrailEdgeActive[m,n] = FALSE initializesearch for trailing edge LastI[m,n] = BlackPoint[m,n] initializeintensity value for k−1 LoopLead If LeadEdgeMidPassed[m,n] = FALSE, Do If i[m,n,k] > MidHeight[m,n] Leading edge crossed midpoint?  LeadEdgeMidPassed[m,n] = TRUE   LeadingCrossover[m,n] = k +{(MidHeight[m,n] − LastI[m,n])/(i[m,n,k] − LastI[m,n])}  TrailEdgeActive[m,n] = TRUE  endif endif If TrailEdgeActive[m,n] =TRUE, Do  If i[m,n,k] < MidHeight[m,n] Trailing edge crossed midpoint?  TrailingCrossover[m,n] = k + {(MidHeight[m,n] − LastI[m,n])/(i[m,n,k]− LastI[m,n])}   TrailEdgeActive[m,n] = FALSE  endif endif k = k+1increment sub-frame counter If k<K, GoTo LoopLead end of CoM[m,n] =TrailingCrossover[m,n] − LeadingCrossover[m,n] m = m + 1 incrementcolumn counter If m ≠ M, GoTo LoopMN end of column? endif m=0 if yes,reset column counter n = n + 1 and increment row counter If n ≠ N, GoToLoopMN end of row? endif if yes, CoM algorothm complete

FIG. 8 illustrates components of an embodiment of asub-frame-processing, composite imaging system. There are threesequential stages of the system—an analog focal plane array 160, anoff-device transfer 162 and A/D conversion, and a digital sub-frameprocessor 164. In embodiments, pixels 166 within a focal plane array 160include a K-bit shift register or other analog logic for storinginformation for K sub-frames. In embodiments, transfer 162 ofinformation off an imaging device like a focal plane array 160, thetransfer 162 of which includes A/D conversion, is specified according toa transfer rate in the form of, typically, multiple gigabytes persecond. In embodiments, digital sub-frame processing 164 will includememory 168 to store the digital information for the K sub-frames, willinclude one or more processing elements like CPUs or GPUs 170, and willinclude digital storage for the one or more composite images 172 thatresult from the sub-frame processing.

The overall throughput and composite image rate for a device isdetermined by the durations of the three stages 162, 162, 164. Inembodiments, a duration for an imaging window establishes the time ittakes for all K sub-frames to be integrated and shifted into analogshift registers located at each pixel 166. In embodiments,considerations for an imaging window duration 174 are determined by theamount of motion expected in a scene, the amount of motion expected fora composite camera, and the desired maximum horizontal and verticalpixel movement for sub-frame zero through sub-frame K−1. In embodiments,an imaging window duration for optimal performance for forward-facingand rear-facing camera automotive applications is in the range from 50μSec to 200 μSec. In embodiments, side-facing or oblique-angleautomotive applications provide optimal performance with imaging windowsdurations in the range from 25 μSec to 150 μSec. In embodiments, smartphone and industrial camera applications provide optimal performancewith imaging windows durations in the range from 50 μSec to 2000 μSec.Transfer duration 176 specifies the time it takes to transfer 162 allsub-frames off an imaging device. Sub-frame transfer duration isdetermined by the focal plane array 160 bus transfer 162 rate and isdefined according to:

TransferDuration=(AD×M×N×K)/(R _(Transfer)×2³⁰×8)  Eq. 14

Where AD is the number of bits utilized in A/D conversion

-   -   M is the number of columns in a focal plane array    -   N is the number of rows in a focal plane array    -   K is the number of sub-frames per composite image    -   R_(Transfer) is the specified transfer rate of a bus in GB/sec    -   2³⁰ represents the number of bytes in a gigabyte    -   8 represents the number of bits in a byte

As an example, the transfer duration 176 for a 16 megapixel compositeimaging system with K=32 sub-frames per composite image is computedaccording to these parameters:

Focal Plane Array size 16,777,216 pixels Bits per pixel for A/DConversion 12 bits/pixel Bytes per pixel 1.5 Bytes/pixel Focal PlaneArray bus transfer rate 5 GB/sec Number of sub-frames per composite 32sub-frames image

The resulting transfer duration according to Eq. 14 is 150 milliseconds(mSec). In embodiments, upon transfer of information to sub-frame memory168, the CPU/GPU 170 performs sub-frame processing at the pixel level todetermine black point, white point, and CoM for each pixel. Inembodiments, utilizing multiple GPUs for processing will typically leadto a lower elapsed time for pixel processing. In embodiments, sub-framepixel processing time for each pixel, expressed in microseconds, iscomputed according to:

t(m,n)_(Sub-framePixelProcessing)=OpNum/MFLOP  Eq. 15

Where OpNum is the number of operations per pixel to perform analgorithm

-   -   MFLOP stands for Mega-FLOPs and is the number of millions of        floating point operations per second for a single GPU

In embodiments, for a CoM algorithm with OpNum equal to 500 operationsrunning on a 50 MFLOP processor, Eq. 15 results in an elapsed time forprocessing of a single pixel of 10 microseconds. In embodiments, for acamera system with M×N pixels and a frame processor that includesmultiple CPU/GPU cores, the processing duration is computed accordingto:

ProcessingDuration=(t(m,n)_(Sub-framePixelProcessing) ×M×N)/NumPU  Eq.16

Where M is the number of columns in a focal plane array

-   -   N is the number of rows in a focal plane array    -   NumPU is the number of processing units used for algorithmic        computation

In embodiments, for a camera system with M equal to 4096 pixels, N equalto 4096 pixels, and a frame processor that includes 1024 CPU/GPU coreswith each core running at 50 MFLOPs, the resulting processing durationis 163.84 milliseconds. Having computed the durations for the stages ofcomposite image collection, transfer, and processing, the overallelapsed time of the stages is:

Imaging Window 0.16 milliseconds Transfer Duration 150 millisecondsProcessing Duration 163.84 milliseconds Total Elapsed Time 314.00milliseconds

The elapsed time of 314.00 milliseconds results in an overallperformance specification for a 32 sub-frame processing, compositeimage-generating 3D+ camera of approximately 3.2 composite images persecond. For applications that require performance of 30 or 60 images persecond, 3.2 images per second comes well short of meeting therequirements. In embodiments, camera architecture may be modified forpipelined processing whereby sequential stages in a process areoverlapped in time by utilizing extra storage and/or additionalelectronics, typically at the expense of higher components costs andhigher electrical current requirements. In embodiments, total elapsedtime for pipelined operation may be reduced to 163.84 milliseconds,which is the elapsed time for the stage with the longest elapsed time.The elapsed time of 163.84 milliseconds results in an overallperformance specification for a 32 sub-frame processing, compositeimage-generating 3D+ camera of approximately 6.1 composite images persecond, which is still well short of a desired throughput rate for manyimaging applications.

According to Wong(https://www.imperial.ac.uk/media/imperial-college/faculty-of-engineering/computing/public/1718-pg-projects/WongM-Analog-Vision.pdf),Focal-Plane Sensor-Processor (FPSP) chips are a special class of imagingdevices in which the sensor arrays and processor arrays are embeddedtogether on the same silicon chip (Zarandy, 2011). Unlike traditionalvision systems, in which sensor arrays send collected data to a separateprocessor for processing, FPSPs allow data to be processed in place onthe imaging device itself. This unique architecture enables ultra-fastimage processing even on small, low-power devices, because costlytransfers of large amounts of data are no longer necessary.

According to Wong, the SCAMP-5 Vision Chip is a Focal-PlaneSensor-Processor (FPSP) developed at the University of Manchester (Careyet al., 2013a). FIG. 9 illustrates a block diagram of a prior artSCAMP-5 FSPS, which comprises 65,536 Processing Elements 192 (PEs)integrated in a 256×256 imager array 190. Each individual PE 192includes a photodetector (pixel) and a processor (ALU, registers,control, and I/O circuits). Processor instructions are common across thedevice, with each individual PE 192 executing the common instructions ontheir own local data. Each PE 192 also has an activity flag, which canbe set as required, allowing for some degree of local autonomy byspecifying instructions to be carried out only by selected Pes 192.These flags can therefore be used to implement conditional operationswhen necessary. Instructions are received from a microcontrollerattached to the chip, which sends a sequence of 79-bit instruction 196words determining the algorithm to be executed. Instructions 196 areexecuted simultaneously across the PE 192 array, allowing instructions196 to be rapidly completed in parallel. Each PE 192 comprises 6 analogregisters (A-F) and 13 digital registers (R0-R12). A key distinguishingfeature of the SCAMP-5 is that, unlike almost all mainstream processorstoday, arithmetic operations are carried out by the analog registers.These operations, including summation, subtraction, division, andsquaring, are implemented using analog current-mode circuits connectedto an analog bus 194 and are able to operate directly on the analogpixel values without a need for analog to digital conversion.

According to Wong, the fully-parallel interface coupled with the use ofanalog registers for arithmetic operations has allowed the SCAMP-5 toachieve superior outcomes on key performance metrics, particularly interms of frame rate and power consumption. The SCAMP-5 architectureallows for the transfer of a complete image frame from the image sensorarray to the processor array in one clock cycle (100 ns), which equatesto a sensor processing bandwidth of 655 GB/s (Martel and Dudek, 2016).This allows for the implementation of vision algorithms at extremelyhigh frame rates which are simply unattainable with traditionalarchitectures. For example, Carey et al. (2013a) demonstrated anobject-tracking algorithm running at 100,000 fps. On the other hand,when operating at lower frame rates, the SCAMP-5 can function atultra-low power consumption rates. Carey et al. (2013b) demonstrated avision system capable of carrying out loiterer detection, which operatedcontinuously at 8 frames per second for 10 days powered by threestandard AAA batteries. These superior performance characteristics havepositioned the SCAMP-5 as an ideal device for implementing visionalgorithms in low-power embedded computing systems (Martel and Dudek,2016).

SCAMP-5 and other FPSP chips are known as neighbor-in-space FPSP devicesbecause they perform operations at the pixel level and will performprocessing within a single frame of data. Each pixel processing elementhas the ability to reference and perform operations for neighboringpixels in space. To this point, sub-frame processing for composite imagecreation has not required neighbor-in-space processing and, as a result,has been unable to benefit significantly from a neighbor-in-space FPSPimplementation. In contrast, sub-frame processing requiresneighbor-in-time processing whereby pixel (m,n) in a sub-frame isprocessed along with pixels (m,n) from other sub-frames within acollection of sub-frames collected within an imaging window for acomposite image or a collection of composite images.

FIG. 10 illustrates a functional diagram for a pixel in aneighbor-in-time analog pixel processing configuration. In embodiments,each pixel in a focal plane array 200 consists of a photodetectorintegration circuit 202, a sub-frame shift register 204 used for thecollection of the K sub-frames, a result shift register 208 that is usedto prepare result data for transfer off the focal plane array 200, anoutput select transistor 209 for enabling bit 0 of the result shiftregister 208 onto an output transfer bus, and analog elements in aneighbor-in-time analog pixel processor 206 (NitAPP). In embodiments,NitAPP 206 refers to the analog processing elements used forneighbor-in-time analog processing and NitAPP Pixel refers to circuitrythat includes NitAPP 206, photodetector integration circuit 202,sub-frame shift register 204, result shift register 208, and outputselect transistor. Instructions for each NitAPP are provided by a J-bitinstruction word provided by an instruction bus 203 from a digitalmicrocontroller 201 (DX).

FIG. 11 illustrates a detailed NitAPP block diagram showing analogstorage and processing elements connected to an analog bus 210. Inembodiments, the NitAPP circuitry is used to execute instructions thatwill operate on a K-bit sub-frame composite image and will determine awhite point value, a black point value, and a trapezoid center of mass(CoM) value. NitAPP elements include a multiplier 228 with three bits ofcontrol, Flag0 212, Flag1 214, and Flag2 216 logic elements that includecompare logic and storage of flag bits used for conditional execution ofinstructions, sub-frame bit 0 read 218 which enables bit 0 of thesub-frame shift register 225 onto the analog bus 210, three registersets 220, 222, 224 enabled by, respectively Flag0 212, Flag 1 214 andFlag 2 216. I_(Count) 226 represents an electrical current source thatis used to implement a sub-frame counter utilizing analog levels. Inembodiments, the only connection to a digital control module or adigital micro-sequencer is through the digital control signals that makeup the J-bit instruction word. In embodiments, all math and logic withinNitAPP is performed using analog electrical current levels and analogelectrical voltage levels.

In embodiments, switched current SI circuitry is used to convey basicfunctionality. In practice, more complex circuitry is used in order toreduce processing errors, to increase accuracy, and to reduce powerdissipation. FIG. 12 illustrates an S²I description of registercircuitry. In embodiments, a functional depiction of register RB0 230 isshown in an exploded view that illustrates all of the transistorcomponents that comprise the register 230. The write operation isperformed in two phases, with a three-transistor phase generator 246provided to split the RB0_Wrt 244 digital signal into two phases for thewrite operation. The input transistor block 238 shows two phase 0transistors and one phase 1 transistor that are enabled by a high levelon the Flag0 234 signal supplied to the gate of each transistor. Anactive RB0_Wrt 244 signal with an active Flag0 234 signal allows theanalog value on the analog bus 232 to be written to the storage portion240 of the register 230. An active high level on the digital RB0_Rd 236signal enables the stored register 230 value onto the analog bus 242. Inembodiments, for the RA0, RB0, RC0, RD0 registers, the input signalsRA0_Wrt, RA0_Rd, RB0_Wrt, RB0_Rd, RC0_Wrt, RC0_Rd, RD0_Wrt and RD0_Rdare digital signals that make up part of the digital instruction wordand Flag0 is an analog signal provided by the Flag0 compare logic.

FIG. 13 illustrates an analog multiplier 250 functional block and a S²Idescription of a multiplier. In embodiments, the multiplier 250circuitry performs analog multiplication between a stored analog valueand an analog value presented by an analog bus 252. In embodiments,multiplier circuitry consists of stored input value circuitry 256,analog multiplication circuitry 262, and result circuitry 264. Inembodiments, an input value is stored when Multi_In_Write 254 isactivated. In embodiments, an output value is stored whenMulti_Out_Write 258 is activated. In embodiments, an output value isenabled onto the analog bus 252 when Multi_In_Enbl 260 is activated. Inembodiments, input value circuitry 256 and output value circuitry 264are shown utilizing three-transistor SI logic. In other embodiments,other known forms of input and output circuitry utilizing S²I, S³I orother storage circuitry constructs for storage of analog values may beutilized.

FIG. 14 illustrates a compare-and-flag 270 functional block and a SIdescription of compare and flag circuitry. In embodiments, the Flag0 270register is implemented as a D-latch. It can be set globally byactivating the Flag0_Set 276 signal. In embodiments, during a comparisoninstruction the Flag0 272 value is charged toward VDD or dischargedtoward ground, depending on the sign of the current from the analog bus278. The Flag0 272 value is stored in the register by activating theFlag0_Latch 274 signal.

FIG. 15 illustrates a current source 280 functional block and a SIdescription of enabling a known current source onto an analog bus. Inembodiments, the current source 282 value is selected at a level that islarge enough to achieve sufficient granularity for determining loopcount values and is small enough to accommodate the accumulations ofmultiple I_(Count) values in a loop counter register. Stated anotherway, the I_(Count) value is selected according to:

I _(Count) <I _(max) /K  Eq. 17

Where I_(max) is the maximum current value for an analog storageregister

K is the number of sub-frames for algorithms that require an analog loopcounter

In embodiments, I_(Count) is enabled onto the analog bus 288 when theICount_Enbl 284 signal is activated. An exemplary analog count circuitfor use in this embodiment may consist of a single stage amplifier witha large capacitive feedback that accumulates a Charge that isproportional to the number of pulses counted for each event enabled bythe Flag0_Set 276 signal. Other examples of analog counter circuits mayalso be used in various embodiments, such as are shown and described inU.S. Pat. No. 7,634,061, the contents of which are hereby incorporatedby reference.

When using a DX for providing instructions to MxN NitAPP elements, allNitAPPs perform the same instruction simultaneously. In embodiments,conditional operations are handled by using information from the Flag0,Flag1, and Flag2 bits, which enable or disable operations for registerbanks. In embodiments, registers are used for storing intermediateresults, are used for event counters, and are used for conditionalinstruction execution based on flag bits. In embodiments, a 32-bitdigital instruction word is routed to each of the MxN NitAPP elements,whereby each instruction bit controls the gate input to a switchingtransistor or controls current flow from the source to the gate for anMOS transistor. In embodiments, the definition of the bits for a 32-bitdigital instruction word is:

NitAPP Instruction Bit # switch Function 0 SFSR_Xfer Sub-frame ShiftRegister Transfer 1 SFSR_Shift Sub-frame Shift Register Bit Shift 2SFSR_PD_Sel Sub-frame Shift Register Photodetector Select 3 SFSR_RdSub-frame Shift Register Bit 0 Enable to Analog Bus 4 Mult_In_Wrt WriteInput Value to Multiplication Block 5 Mult_Out_Wrt Write Output Valuefrom Multiplication Block 6 Mult_Out_Rd Enable Multiplication Output toAnalog Bus 7 Flag0_Latch Latch Flag0 Based on Compare Circuit 8Flag0_Set Set Flag0 9 Flag1_Latch Latch Flag1 Based on Compare Circuit10 Flag1_Set Set Flag1 11 Flag2_Latch Latch Flag2 Based on CompareCircuit 12 Flag2_Set Set Flag2 13 RA0_Rd Enable Register A0 to AnalogBus 14 RA0_Wrt Write Analog Bus Value to Register A0 15 RB0_Rd EnableRegister B0 to Analog Bus 16 RB0_Wrt Write Analog Bus Value to RegisterB0 17 RC0_Rd Enable Register C0 to Analog Bus 18 RC0_Wrt Write AnalogBus Value to Register C0 19 RD0_Rd Enable Register D0 to Analog Bus 20RD0_Wrt Write Analog Bus Value to Register D0 21 RA1_Rd Enable RegisterA1 to Analog Bus 22 RA1_Wrt Write Analog Bus Value to Register A1 23RB1_Rd Enable Register B1 to Analog Bus 24 RB1_Wrt Write Analog BusValue to Register B1 25 RA2_Rd Enable Register A2 to Analog Bus 26RA2_Wrt Write Analog Bus Value to Register A2 27 RB2_Rd Enable RegisterB2 to Analog Bus 28 RB2_Wrt Write Analog Bus Value to Register B2 29ICount_Enbl Enable Icount Current to Analog Bus 30 Result_SR_Xfer ResultShift Register Transfer 31 Result_SR_Shift Result Shift Register BitShift

In embodiments, pseudocode for DX instructions that perform black point,white point, and CoM computations for each pixel, along with theassociated NitAPP instruction values, is shown below:

DuC Pseudocode Constant s: K = number of sub-frames per composite imageM = number of columns of pixels in FPA N = number of rows of pixels inFPA DuC NitAPP DuC instruction Comment NitAPP Inst Switches NitAPPComments initialize loop count for k = 0 BP, WP Initialize Flag0[m,n]Set Flag0 Flag0_Set Enable Flag0 Registers Initialize Flag1[m,n] SetFlag1 Flag1_Set Enable Flag1 Registers Initialize Flag2[m,n] Set Flag2Flag2_Set Enable Flag2 Registers initialize BP to a high Mult_In_Wrt,BlackPoint[m,n] = 0x3FF value Icount −> Mult Icount_Enbl Icount to Multinput Mult * Icount −> Mult_Out_Wr Mult t, Icount_Enbl Mult = Icountsquared Mult_Out_Rd, Mult −> RC0 RC0_Wrt RC0 = BlackPoint[m,n]initialize WP to a low RA0_Wrt, WhitePoint[n,m] = 0 value Icount −> RA0Icount_Enbl RA0 = Icount − Icount RA0_Rd, RA0 + Icount −> Icount_Enbl,RD0 RD0_Wrt RB2 = WhitePoint[m,n] read bit K from shift LoopWP Readi[m,n,k] register bit k Flag0_Latch, lowest SFSR_Rd, If i[m,n,k] <BlackPoint[m,n] so far? SFSR < RC0 RC0_Rd Check for new BlackPoint ifyes, make bit BlackPoint[m,n] k new SFSR_Rd, Conditional BlackPoint =i[m,n,k] lowest SFSR −> RC0 RC0_Wrt update endif Set Flag0 Flag0_SetEnable Flag0 Registers bit k Flag0_Latch, If i[m,n,k] > highest SFSR_Rd,WhitePoint[m,n] so far? SFSR > RD0 RD0_Rd Check for new WhitePoint ifyes, make bit  WhitePoint[m,n] = k new SFSR_Rd, Conditional WhitePointi[m,n,k] highest SFSR −> RD0 RB2_Wrt update endif Set Flag0 Flag0_SetEnable Flag0 Registers increme nt sub- frame k = k+1 counter SFSR_XferSFSR circular transfer circular shift of K Circular Shift of ShiftSFShiftRegister[m,n] shift SFSR SFSR_Shift SFSR circular shift registerIf k<K, GoTo LoopWP end of Result_SR_Xf Send BlackPoint[m,n] to RC0 −>Result_SR er, RC0_Rd output shift register Result_SR_Shi Shift Output SRft Result_SR_Xf Send WhitePoint[m,n] to RD0 −> Result_SR er, RD0_Rdoutput shift register Result_SR_Shi Shift Output SR ft mid-MidHeight[m,n] = height (WhitePoint[m,n] − intensity RC0_Rd,BlackPoint[m,n]) / 2 value RC0 −> RB0 RB0_Wrt Negate BlackPoint[m,n]RC0_Wrt, RA0_Wrt, RC0 = (RB0 + RB0_Rd, RD0)/2 RD0_Rd RC0 =MidHeight[m,n] initialize leading Flag1 = LeadEdgeMidPassed[m,n] = edgeLeadEdgeMidPassed[m,n] FALSE CoM flag Set Flag1 Flag1_Set = FALSEinitialize loop count for k = 0 CoM initialize NitAPP SF Icount_Enbl,SFCount[m,n] = 0 counter Icount −> RA0 RA0_Wrt Icount_Enbl, RD0 =Icount + RD0_Wrt, RA0 RA0_Rd RD0 = SFCount[m,n] = 0 initializeTrailEdgeActive[m,n] = search for Flag2_Latch, Flag2 = FALSE trailingedge RA0 > 0 RA0_Rd TrailEdgeActive[m,n] initialize intensity valueRC0_Rd, LastI[m,n] = BlackPoint[m,n] for sf−1 RC0 −> RA2 RA2_Wrt RA2 =LastI[m,n] LoopLea d If LeadEdgeMidPassed[m,n] = FALSE, Do Leading edge If i[m,n,k] > crossed RC0_Rd, MidHeight[m,n] midpoint? RC0 −> RA0RA0_Wrt Negate MidHeight[m,n] Flag0_Latch, SFSR_Rd, SFSR > RA0 RA0_RdIcount_Enbl,   LeadEdgeMidPassed[m,n] =TRUE Icount −> RA0 RA0_WrtFlag1_Latch, LeadEdgeMidPassed[m,n] RA0 > 0 RA0_Rd = TRUE  TrailEdgeActive[m,n] = TRUE Set Flag2 Flag2_Set  LeadingCrossover[m,n] = SFCount[m,n] + I_(Count)*{(MidHeight[m,n] −LastI[m,n])/(i[m,n,k] − LastI[m,n])} RA2_Rd, RA2 −> RA0 RA0_Wrt NegateLastI[m,n] RB0_Wrt, SFSR_Rd, RB0 = SFSR + RA0 RA0_Rd RB0 = SFSR −LastI[m,n] RB0_Rd, RB0_Wrt, Mult_In_Wrt, Invert RB0 Mult_Out_Wrt,Mult_Out_Rd RB0_Rd, RB0 −> Mult_In Mult_In_Wrt RB2_Wrt, RA1_Rd, RB2 =MidHeight[m,n] − RB2 = RA1 + RA0 RA0_Rd LastI[m,n] RB2_Rd, RB0 = Icount*Mult_Out_Wr (RB2/RB0) t, Icount_Enbl RB1_Wrt, RD0_Rd, RB1 = RD0 + RB0RB0_Rd  endif endif If TrailEdgeActive[m,n] = TRUE, Do Trailing edge  Ifi[m,n,k] < crossed SFSR_Rd, MidHeight[m,n] midpoint? SFSR −> RA0 RA0_WrtNegate SFSR Flag1_Latch, SFSR_Enbl, RA0 > RC0 RC0_Rd  TrailingCrossover[m,n] = SFCount[m,n] + I_(Count)*{(MidHeight[m,n] −RA2_Rd, LastI[m,n])/(i[m,n,k] − LastI[m,n])} RA2 −> RA0 RA0_Wrt NegateLastI[m,n] RB0_Wrt, SFSR_Rd, RB0 = SFSR + RA0 RA0_Rd RB0 = SFSR −LastI[m,n] RB0_Rd, RB0_Wrt, Mult_In_Wrt, Invert RB0 Mult_Out_Wrt,Mult_Out_Rd RB0_Rd, RB0 −> Mult_In Mult_In_Wrt RB2_Wrt, RA1_Rd, RB2 =MidHeight[m,n] − RB2 = RA1 + RA0 RA0_Rd LastI[m,n] RB2_Rd, RB0 =Icount * Mult_Out_Wr (RB2/RB0) t, Icount_Enbl RB2_Wrt, RD0_Rd, RB2 =RD0 + RB0 RB0_Rd Icount_Enbl,   TrailEdgeActive[m,n] = FALSE Icount −>RA0 RA0_Wrt Flag2_Latch, LeadEdgeMidPassed[m,n] RA0 > 0 RA0_Rd = TRUE endif endif increme nt sub- frame k= k+1 counter SFSR_Xfer SFSRcircular transfer circular shift of K shift Circular Shift of ShiftSFShiftRegister[m,n] register SFSR SFSR_Shift SFSR circular shiftincreme nt sub- SFCount[m,n] = frame RD0_Rd, SFCount[m,n] + I_(Count)counter RD0 −> RA0 RA0_Wrt Negate SFCount[m.n] RB0_Wrt, RB0 = RA0 +RA0_Rd, Icount Icount_Enbl RB0_Rd, RB0 −> RD0 RD0_Rd If k<K, GoToLoopLead end of CoM[m,n] = TrailingCrossover[m,n] − NegateLeadingCrossover[m,n] RB1 −> RA0 LeadingCrossover[m,n] RA1 = RA0 + RB2RA1 = CoM[m,n] Result_SR_Xf Send CoM[m,n] to output RA1 −> Result_SR er,RA1_Rd shift register Result_SR_Shi Shift Output SR ft

FIG. 16 illustrates components of embodiments for asub-frame-processing, composite imaging system 290 that utilizes NitAPPprocessing. The system 290 includes a mixed signal focal plane array 292front end and a digital sub-frame-processing back end. In embodiments,digital elements include NitAPP result sub-frame memory 304 for storingdigitally converted results from the NitAPP processing, CPU/GPU 306 fordigital result processing for creating final images, and composite imagememory for storing the results of composite image 308 creation. Inembodiments, a focal plane array 294 consists of M×N pixels, where M isthe number of columns and N is the number of rows of photodetectorelements. In embodiments, each pixel consists of a NitAPP(m,n) element300 that includes a photodetector, photodetector control circuitry, andNitAPP processing circuitry. In embodiments, a digital microcontroller296 executes instructions that determine the algorithmic processingfunctions that are simultaneously performed at each NitAPP(m,n) element300. In embodiments, NitAPP functions are controlled via a 32-bitinstruction bus 298. In embodiments, analog information that is readfrom the result registers of the NitAPP(m,n) elements 300 is convertedto digital form via one or more A/D converters 302 prior to transmittingfrom the focal plane array 292 to the digital back end.

In embodiments, there are four sequential time durations of the system290—an analog focal plane array imaging window 310, an on-FPAcomputation duration for NitAPP processing 312, an off-chip transfer 314and A/D conversion, and a digital processing duration 316. The overallthroughput and composite image rate for a device is determined by thedurations of the four stages 310, 312, 314 and 316. In embodiments, aduration for an imaging window establishes the time it takes for all Ksub-frames to be integrated and shifted into analog shift registerslocated at each NitAPP pixel 300. In embodiments, considerations for animaging window duration 174 are determined by the amount of motionexpected in a scene, the amount of motion expected for a compositecamera, and the desired maximum horizontal and vertical pixel movementfor from sub-frame zero through sub-frame K−1. In embodiments, animaging window of 160 μSec for automotive applications meets thesub-frame horizontal and vertical alignment guidelines forforward-facing and rear-facing camera applications. In embodiments,NitAPP processing is the amount of time required for the DuC 296 toissue all of the instructions to the NitAPP[m,n] elements for thedesired algorithmic processing and control functionality for on-pixel,sub-frame processing. In embodiments, transfer duration 314 specifiesthe time it takes to read result information from all pixels andtransfer all sub-frames off a focal plane array 292 and into NitAPPresult memory 304. In embodiments, processing duration 316 is the timeit takes to digitally produce composite images 308 from the informationcontained in the NitAPP result memory 304.

In embodiments, NitAPP architecture displays significant throughputadvantages versus digital sub-frame-processing systems. As an example, athroughput comparison is presented for NitAPP-processed anddigitally-processed images for a 16 megapixel composite imaging systemwith K=32 sub-frames per composite image according to these parameters:

Focal Plane Array size 16,777,216 pixels Bits per pixel for A/DConversion 12 bits/pixel Bytes per pixel 1.5 Bytes/pixel Focal PlaneArray bus transfer rate 5 GB/sec Number of sub-frames per composite 32sub-frames image

In embodiments, the duration comparisons are made for comparativealgorithms to determine black point, white point and center of mass(CoM) for each of the 16 megapixels. The overall durations for twoimaging systems are:

Digital Sub-frame NitAPP Imaging Window 0.16 ms 0.16 ms On-FPAComputation 0 ms 0.10 ms Transfer Duration 150 ms 14.06 ms  ProcessingDuration 163.84 ms 3.28 ms Total Elapsed Time 314.0 ms 17.6 msImages/second - no pipeline 3.2 56.7 Images/second - pipelined 6.1 71.1

In embodiments, on-FPA computation for NitAPP consists of the durationrequired to execute the NitAPP instructions for a black point, whitepoint and CoM algorithm. The duration (in μSec) of the algorithm iscomputed according to:

t(m,n)_(NitAPP)=OpNum_(NitAPP)/MFLOP_(NitAPP)  Eq. 18

Where OpNum_(NitAPP) is the number of NitAPP instructions to perform analgorithm

-   -   MFLOP_(NitAPP) stands for Mega-FLOPs and is the number of        millions of floating point operations per second for a single        NitAPP processing element

For the WP/BP/CoM algorithm presented herein, the number of NitAPPoperations is 1012, which is 9 instructions for BP/WP start, 256instructions (8 instructions times 32 loops) for the WP/BP loop, 4instructions for WP/BP end, 736 instructions (23 instructions times 32loops) for the CoM loop, and 7 for CoM end. Utilizing a NitAPPinstruction clock of 10 MHz results in a MFLOP_(NitAPP) equal to 10. Eq.17 results in an on-FPA computation time of 101.2 μs. For a camerasystem with M equal to 4096 pixels, N equal to 4096 pixels, and a frameprocessor that includes 1024 CPU/GPU cores with each core running at 50MFLOPs, the digital sub-frame processing duration is 163.84 millisecondsbased on a per-pixel algorithm of 500 instructions. When utilizingNitAPP for on-FPA processing, the digital back end has a reducedprocessing duration because fewer instructions are required per pixel.In embodiments, if the digital processing back end requires 10instructions per pixel to perform composite image creation, the CPU/GPUprocessing duration is reduced to 3.28 milliseconds.

The elapsed time of 17.6 milliseconds for NitAPP sub-frame processingresults in an overall performance specification for a 32 sub-frameprocessing, composite image-generating 3D camera of approximately 56.7composite images per second. For applications that require performanceof 30 images per second, 56.7 images per second more than meets therequirements. In embodiments, camera architecture may be modified forpipelined processing whereby sequential stages in a process areoverlapped in time by utilizing extra storage and/or additionalelectronics, typically at the expense of higher components costs andhigher electrical current requirements. In embodiments, total elapsedtime for pipelined operation for NitAPP processing may be reduced to14.06 milliseconds, which is the elapsed time for the stage with thelongest elapsed time. The elapsed time of 14.06 milliseconds results inan overall performance specification for a 32 sub-frame processing,composite image-generating 3D camera of approximately 71.1 compositeimages per second, which is sufficient for meeting the throughput ratefor 60 image-per-second imaging applications.

Digital CPUs and GPUs typically attempt to extract top performance outof a given technology, often at the expense of power consumption. Theuse of NitAPP processing for composite image creation offers the benefitof lower overall device power consumption because most of the processingis shifted from power-hungry digital processing to very-low-power analogcomputing. Utilizing a 10 nm feature size silicon fabrication process,the power consumption for various elements can be expressed as:

Function Power Units Photodetector accumulation 80 pW per accumulationper pixel NitAPP instruction 6 pW per NitAPP element Digital MemoryRead/Write 0.45 nW per byte GPU instruction 0.08 nW per instruction FPAtransfer and A/D 0.85 nW per byte Conversion

In embodiments, a power consumption comparison for digital sub-frameprocessing and for NitAPP sub-frame processing for a sixteen megapixel,32 sub-frame composite image utilizing a 10 nm process is:

Digital NitAPP #/img/ Digital #/img/ NitAPP Function pixel mW/imagepixel mW/image FPA accumulations 1 1.3 1 1.3 NitAPP instructions 0 01012 101.9 FPA Xfer and A/D 48 684.5 4.5 64.2 Digital Read/Write 96724.8 9 67.9 GPU instructions 500 671.1 10 13.4 Total Power (mW) 2081.7248.8

FIG. 17 illustrates a scene that is imaged by a sub-frame compositeimage camera 334. Objects in the scene consist of a stationarybackground object 330 and a foreground object 332. In embodiments,sub-frame processing is utilized to detect and quantify three-axismotion for the foreground object 332 whereby the three axes of motionare determined relative to the camera 334. Camera axis m 336 indicatesthe direction of motion for increasing values of horizontal camera 334pixels, camera axis n 338 indicates the direction of motion forincreasing values of vertical camera 334 pixels, and camera axis d 340indicates the direction of motion for increasing values of distancebetween a camera pixel and an object 330, 332. In embodiments, asub-frame trapezoid descriptor of 32/8/8/12/1/5 is used forelectro-optical sub-frame camera parameters, and intensity vs. sub-frameplots 342 are shown for a 3×3 pixel grouping that corresponds to pixelsthat represent the upper right corner of a foreground object 332.Trapezoidal analysis for a 3×3 pixel grouping yields the trapezoidalparameters shown in Table 1 below:

TABLE 1 m − 1 m m + 1 n + 1 I_(bp-le): 80 I_(bp-le): 80 I_(bp-le): 80I_(bp-te): 68 I_(bp-te): 68 I_(bp-te): 74 I_(wp-le): 256 I_(wp-le): 256I_(wp-le): 256 I_(wp-te): 224 I_(wp-te): 224 I_(wp-te): 240 SF_(le-mid):12.5 SF_(le-mid): 12.5 SF_(le-mid): 12.5 SF_(te-mid): 24.5 SF_(te-mid):24.5 SF_(te-mid): 24.5 n I_(bp-le): 40 I_(bp-le): 40 I_(bp-le): 80I_(bp-te): 34 I_(bp-te): 34 I_(bp-te): 68 I_(wp-le): 128 I_(wp-le): 128I_(wp-le): 256 I_(wp-te): 112 I_(wp-te): 112 I_(wp-te): 224 SF_(le-mid):6.5 SF_(le-mid): 6.5 SF_(le-mid): 12.5 SF_(te-mid): 18.3 SF_(te-mid):18.3 SF_(te-mid): 24.5 n − 1 I_(bp-le): 40 I_(bp-le): 40 I_(bp-le): 80I_(bp-te): 40 I_(bp-te): 34 I_(bp-te): 68 I_(wp-le): 120 I_(wp-le): 128I_(wp-le): 256 I_(wp-te): 120 I_(wp-te): 112 I_(wp-te): 224 SF_(le-mid):6.5 SF_(le-mid): 6.5 SF_(le-mid): 12.5 SF_(te-mid): 18.3 SF_(te-mid):18.3 SF_(te-mid): 24.5

Based on trapezoidal analysis, a slope is computed for each pixel forthe white point portion of the trapezoid according to:

Slope_(wp)(m,n)=[I _(te-wp)(m,n)−I _(le-wp)(m,n)]/ΔSF_(trapezoid-top)  Eq.19

Where I_(te-wp)(m,n) is the trailing edge white point intensity valuefor pixel (m,n)

-   -   I_(le-wp)(m,n) is the leading edge white point intensity value        for pixel (m,n)    -   ΔSF_(trapezoid-top) is the width of the top of a trapezoid in #        of sub-frames

White point intensity value analysis for a 3×3 pixel grouping yields thewhite point slope values shown in Table 2 below, along with computeddistances for each pixel in accordance with

TABLE 2 m − 1 m m + 1 n + 1 Slope_(wp): −8 Slope_(wp): −8 Slope_(wp): −4Distance: 22.18 m Distance: 22.18 m Distance: 22.18 m n Slope_(wp): −4Slope_(wp): −4 Slope_(wp): −8 Distance: 14.87 m Distance: 14.87 mDistance: 22.18 m n − 1 Slope_(wp): 0 Slope_(wp): −4 Slope_(wp): −8Distance: 14.87 m Distance: 14.87 m Distance: 22.18 m

In embodiments, pixels (m−1, n−1), (m, n−1), and (m+1, n−1) form anm-motion pixel triplet whereby motion is detected along the m-axis dueto the zero slope for pixel (m−1, n−1) and non-zero slopes for pixels(m, n−1) and (m+1, n−1) whereby the signs of the non-zero slopes are thesame. The m-motion pixel triplet is the result of an object of highintensity migrating from the field of view (FOV) of pixel (m+1, n−1)into the FOV of pixel (m, n−1), or the m-motion pixel triplet is theresult of an object of low intensity migrating from the field of view(FOV) of pixel (m, n−1) into the FOV of pixel (m+1, n−1). Inembodiments, pixels (m−1, n+1), (m−1, n), and (m−1, n−1) form ann-motion pixel triplet whereby motion is detected along the n-axis dueto the zero slope for pixel (m−1, n−1) and non-zero slopes for pixels(m−1, n) and (m−1, n+1) whereby the signs of the non-zero slopes are thesame. The n-motion pixel triplet is the result of an object of highintensity migrating from the FOV of pixel (m−1, n+1) into the FOV ofpixel (m−1, n), or the n-motion pixel triplet is the result of an objectof low intensity migrating from the field of view (FOV) of pixel (m−1,n) into the FOV of pixel (m−1, n+1). In embodiments, the amplitude ofthe m-motion or the n-motion is computed by determining the sub-framenumber at which the extrapolated high-intensity white point trapezoidslope crosses over the trailing edge black point intensity value for theother non-zero slope pixel in the m-motion or n-motion pixel tripletaccording to Eq. 20 below:

ΔSF _(motion)=[(I _(le-wp)(m,n)—I _(le-wp)(m,n))*(I _(te-bp)(m,n)—I_(le-wp)(m,n))]/ΔSF _(trapezoid-top)

-   -   Where I_(te-wp)(m,n) is the trailing edge white point intensity        value for the high-intensity pixel of a non-zero-sloped pixel        triplet        -   I_(le-wp)(m,n) is the leading edge white point intensity            value for the high-intensity pixel of a non-zero-sloped            pixel triplet        -   I_(te-bp)(m,n) is the trailing edge black point intensity            value for the low-intensity pixel of a non-zero-sloped pixel            triplet        -   ΔSF_(trapezoid-top) is the width of a trapezoid top,            expressed in # of sub-frames, as determined from the derived            parameters of a trapezoid descriptor

In embodiments, m-axis or n-axis motion is expressed as the number ofsub-frame periods required for the intensity value of a pixel tocompletely replace the intensity value of a neighboring pixel thatshares a white point slope sign within a pixel triplet. In embodiments,the amplitude of m-axis or n-axis movement is converted to a length bydetermining the distance of the in-motion object from the camera andutilizing the angular offset between FOVs of neighboring pixels and iscomputed by:

Motion_(m-axis)(m,n)=d(m,n)*sin Δφ(m,n)  Eq. 21

Where d(m,n) is the distance to the nearest pixel of an m-axis triplet

-   -   Δφ(m,n) is the angular offset between the centers of m-axis FOVs

Motion_(n-axis)(m,n)=d(m,n)*sin Δθ(m,n)  Eq. 22

Where d(m,n) is the distance to the nearest pixel of an n-axis triplet

-   -   Δθ(m,n) is the angular offset between the centers of m-axis FOVs

In embodiments, m-axis and n-axis motion is determined according to theidentification of m-axis pixel triplets and n-axis pixel triplets. Theamplitude of m-axis and n-axis motion is determined for same-signed,non-zero-sloped pixel pairs within pixel triplets. The determination ofsign (indicating direction of motion) of the m-axis or n-axis motion ona pixel basis depends on a distance difference between same-signed,non-zero-sloped pixels. In embodiments, the direction of m-axis orn-axis movement is selected according to determining that the pixel withthe shortest distance value is a pixel located on the in-motion objectin a scene. Therefore, the direction of m-axis or n-axis movement willbe from the pixel with the smaller distance parameter to the pixel withthe larger distance parameter.

In embodiments, for m-axis and n-axis motion whereby the same-slopepixel values are at the same distance from the sensor, the pixels likelyrepresent differing intensity values from the same in-motion object.Therefore, the amplitude of the motion is determinable from pixeltriplet processing, but the direction of the movement is determined fromtriplet processing for an in-motion triplet that is nearby in spacewhereby the distances of same-slope pixels are different.

In embodiments, motion in the d axis is determined by computing thewidth of a trapezoid as determined by the distance (in sub-frames)between a leading edge midpoint and a trailing edge midpoint andcomparing it to the width of an ideal trapezoid for a non-moving object.Pixels associated with objects moving toward a sub-frame processing,composite image camera will exhibit trapezoid widths that are less thanthe width of an ideal trapezoid, and pixels associated with objectsmoving away from a sub-frame processing, composite image camera willexhibit trapezoid widths that are greater than the width of an idealtrapezoid. D-axis motion is computed according to Eq. 23 below:

Motion_(d-axis)(m,n)=({[SF _(te-mid)(m,n)−SF _(te-mid)(m,n)]−SF_(mid-height-width) }*C*P _(emitter))/2

-   -   Where SF_(te-mid)(m,n) is the sub-frame for the trailing edge        midpoint for pixel (m,n)        -   SF_(le-mid)(m,n) is the sub-frame for the leading edge            midpoint for pixel (m,n)        -   SF_(mid-height-width) is the width, in number of sub-frames,            at the mid-height of an ideal trapezoid        -   C is a constant for the speed of light in a medium        -   P_(emitter) is the emitter clock period

In embodiments, d-axis motion is determined on a pixel basis and is notdependent on neighbor-in-space intensity values or neighbor-in-spacedistance values. Said another way, d-axis motion is detectable andmeasureable for each pixel in a sub-frame processing, composite imagingsystem.

In embodiments, sub-frame processing in a composite imaging systeminterprets sub-frame intensity values to determine, within a singlecomposite image, pixel parameters like intensity, radiance, luminance,distance, m-axis motion (horizontal motion relative to the sensor),n-axis motion (vertical motion relative to the sensor) and d-axis motion(relative motion toward or away from the sensor). In embodiments, sensorpixel parameters are determined from sub-frame intensity waveformparameter analysis according to Table 3 below:

TABLE 3 Min. Waveform Pixel Waveform Type Sub-frames ParametersProperties WP/BP 2 I_(bp)(m, n) Luminance I_(wp)(m, n) Radiance WP/BP 3I_(le-wp)(m, n) Luminance I_(te-wp)(m, n) Radiance I_(bp)(m, n) m-axismotion n-axis motion Trapezoid 5 I_(wp)(m, n) Luminance I_(bp)(m, n)Radiance CoM(m, n) Distance Trapezoid 6 I_(le-wp)(m, n) LuminanceI_(te-wp)(m, n) Radiance I_(bp)(m, n) Distance CoM(m, n) m-axis motionn-axis motion Trapezoid 8 I_(le-wp)(m, n) Luminance I_(te-wp)(m, n)Radiance I_(bp)(m, n) Distance SF_(le-mid)(m, n) m-axis motionSF_(te-mid)(m, n) n-axis motion d-axis motion Non-overlapping Range 3I₀(m, n) Luminance Gating I₁(m, n) Radiance I_(G-1)(m, n) DistanceeXtended Dynamic 3 I₀(m, n) XDR Intensity Range I₁(m, n) Fill Rate I₂(m,n)

Trapezoidal sub-frame collection and subsequent trapezoid parameterdetermination place high demands on digital-only processing systems. Inembodiments, NitAPP architecture displays significant throughputadvantages versus digital sub-frame-processing systems. As an example, athroughput comparison is presented for NitAPP-processed anddigitally-processed images for a 16 megapixel composite imaging systemwith K=32 sub-frames per composite image according to these parameters:

Focal Plane Array size 16,777,216 pixels Bits per pixel for A/DConversion 12 bits/pixel Bytes per pixel 1.5 Bytes/pixel Focal PlaneArray bus transfer rate 5 GB/sec Number of sub-frames per composite 32sub-frames image

In embodiments, the duration comparisons are made in Table 4 below forcomparative algorithms to determine luminance, radiance, distance,m-axis motion, n-axis motion, and d-axis motion, all within a singlecomposite image, for each of the 16 megapixels.

TABLE 4 Elapsed Time - Elapsed Time - Function Digital NitAPP/DigitalSub-frame Capture (32 sub-frames) 0.16 ms 0.16 ms Compute NitAPPI_(wp-le)(m, n) — 0.02 ms Compute NitAPP I_(wp-te)(m, n) — 0.02 msCompute NitAPP I_(bp)(m, n) — 0.02 ms Determine NitAPP SF_(le-mid)(m, n)— 0.06 ms Determine NitAPP SF_(le-mid)(m, n) — 0.06 ms Transfer NitAPPsub-frames from — 23.43 ms FPA (5 sub-frames) Transfer all sub-framesfrom 150 ms — FPA (32 sub-frames) Compute Digital I_(wp-le)(m, n) 32 ms— Compute Digital I_(wp-te)(m, n) 32 ms — Compute Digital I_(bp)(m, n)32 ms — Determine Digital SF_(le-mid)(m, n) 72 ms — Determine DigitalSF_(le-mid)(m, n) 72 ms — Determine Slope_(wp)(m, n) 4 ms 4 ms DetermineDistance(m, n) 4 ms 4 ms Compute Luminance 4 ms 4 ms Compute Radiance 4ms 4 ms Determine m-axis motion 4 ms 4 ms Determine n-axis motion 4 ms 4ms Determine d-axis motion 4 ms 4 ms Total Elapsed Time 428.16 ms 51.77ms Composite Images per Second 2.34 19.32

Digital CPUs and GPUs typically attempt to extract top performance outof a given technology, often at the expense of power consumption. Theuse of NitAPP processing for composite image creation offers the benefitof lower overall device power consumption because most of the processingis shifted from power-hungry digital processing to very-low-power analogcomputing. Utilizing a 10 nm feature size silicon fabrication process,the power consumption for various elements can be expressed as:

Function Power Units Photodetector accumulation 80 pW per accumulationper pixel NitAPP instruction 6 pW per NitAPP element Digital MemoryRead/Write 0.45 nW per byte GPU instruction 0.08 nW per instruction FPAtransfer and A/D 0.85 nW per byte Conversion

In embodiments, a power consumption comparison for digital sub-frameprocessing and for NitAPP sub-frame processing for a sixteen megapixel,32 sub-frame composite image utilizing a 10 nm process is shown in Table5 below.

TABLE 5 Power Usage - Power Usage - Function Digital NitAPP/DigitalSub-frame Capture (32 sub-frames) 1.3 mW 1.3 mW Compute NitAPPI_(wp-le)(m, n) — 20.37 mW Compute NitAPP I_(wp-te)(m, n) — 20.37 mWCompute NitAPP I_(bp)(m, n) — 20.37 mW Determine NitAPP SF_(le-mid)(m,n) — 62.84 mW Determine NitAPP SF_(le-mid)(m, n) — 62.84 mW TransferNitAPP sub-frames from — 45.86 mW FPA (5 sub-frames) Transfer allsub-frames from 684.5 mW — FPA (32 sub-frames) Compute DigitalI_(wp-le)(m, n) 279.2 mW — Compute Digital I_(wp-te)(m, n) 279.2 mW —Compute Digital I_(bp)(m, n) 279.2 mW — Determine Digital SF_(le-mid)(m,n) 837.5 mW — Determine Digital SF_(le-mid)(m, n) 837.5 mW — DetermineSlope_(wp)(m, n) 34.9 mW 34.9 mW mW Determine Distance(m, n) 34.9 mW34.9 mW mW Compute Luminance 34.9 mW 34.9 mW mW Compute Radiance 34.9 mW34.9 mW mW Determine m-axis motion 34.9 mW 34.9 mW mW Determine n-axismotion 34.9 mW 34.9 mW mW Determine d-axis motion 34.9 mW 34.9 mW mWTotal Power per Composite Image 3442.7 mW 470.25 mW

Table 3 identifies a WP/BP waveform with a minimum of three sub-frames.In embodiments, a minimum of three sub-frames enables the determinationof m-axis and n-axis motion within a single composite image. As anexample of an embodiment, a WP/BP descriptor of 3/50, signifying a whitepoint sub-frame followed by a black point sub-frame followed by a secondwhite point sub-frame with an elapsed time from the start of onesub-frame to the start of a subsequent sub-frame defined as 50 μSec. Inembodiments, Eq. 18 is modified by replacing ΔSF_(trapezoid-width) withΔSF_(wp), and a white point slope is computed for each pixel accordingto:

Slope_(wp)(m,n)=[I _(te-wp)(m,n)−I _(le-wp)(m,n)]/ΔSF _(wp)  Eq. 24

Where I_(te-wp) (m,n) is the trailing edge white point intensity valuefor pixel (m,n)

-   -   I_(le-wp)(m,n) is the leading edge white point intensity value        for pixel (m,n)    -   ΔSF_(wp) is the # of sub-frames between the leading edge and        trailing edge white point sub-frames

In embodiments, the amplitude of the m-motion or the n-motion iscomputed by determining the sub-frame number at which the extrapolatedhigh-intensity white point trapezoid slope crosses over the trailingedge black point intensity value for the other non-zero slope pixel inthe m-motion or n-motion pixel triplet. Eq. 19 is modified by replacingΔSF_(trapezoid-width) with ΔSF_(wp), and the amplitude of m-axis orn-axis motion, expressed in terms of the # of sub-frames, is computedfor each pixel according to according to Eq. 25 below:

ΔSF _(motion)=[(I _(te-wp)(m,n)−I _(le-wp)(m,n))*(I _(te-bp)(m,n)−I_(le-wp)(M,n))]/ΔSF _(wp)

-   -   Where I_(te-wp)(m,n) is the trailing edge white point intensity        value for the high-intensity pixel of a non-zero-sloped pixel        triplet        -   I_(le-wp)(m,n) is the leading edge white point intensity            value for the high-intensity pixel of a non-zero-sloped            pixel triplet        -   I_(te-bp)(m,n) is the trailing edge black point intensity            value for the low-intensity pixel of a non-zero-sloped pixel            triplet        -   ΔSF_(wp) is the # of sub-frames between the leading edge and            trailing edge white point sub-frames

Table 3 identifies a non-overlapping range gating waveform with aminimum of three sub-frames. In embodiments, a minimum of threesub-frames enables the determination of radiance, luminance, anddistance within a single composite image. FIG. 18 illustrates theoptical timing parameters for a non-overlapping range gating descriptorof 3/100/1/2/0. In embodiments, emitter and detector timing arereferenced from an emitter clock 350. In accordance with thenon-overlapping range gating descriptor, the sub-frame 0 detector 352activates when the emitter activates and deactivates 100 nSec after theemitter deactivates, the sub-frame 1 detector activates 100 nSec afterthe emitter activates and deactivates 200 nSec after the emitterdeactivates, and the sub-frame 2 detector activates 200 nSec after theemitter activates and deactivates 300 nSec after the emitterdeactivates.

The Sub-frame 0 graph 354 illustrates the amount of emitter and detectoroverlap for various distances throughout the device range and signifiesthat: 1) emitter and detector experience 100% overlap for distancesbetween 0 and 15 meters 360, 2) emitter and detector overlap decreaseslinearly from 100% to 0% for distances between 15 and 30 meters 362, 3)emitter and detector overlap is 0% for distances between 30 and 45meters 364, and 4) emitter and detector overlap is 0% for distancesbetween 45 and 60 meters 366. The Sub-frame 1 graph 356 illustrates theamount of emitter and detector overlap for various distances throughoutthe device range and signifies that: 1) emitter and detector overlapincreases linearly from 0% to 100% for distances between 0 and 15 meters360, 2) emitter and detector experience 100% overlap for distancesbetween 15 and 30 meters 362, 3) emitter and detector overlap decreaseslinearly from 100% to 0% for distances between 30 and 45 meters 364, and4) emitter and detector overlap is 0% for distances beyond 45 meters366. The Sub-frame 2 graph 356 illustrates the amount of emitter anddetector overlap for various distances throughout the device range andsignifies that: 1) emitter and detector overlap is 0% to 100% fordistances between 0 and 15 meters 360, 2) emitter and detector overlapincreases linearly from 0% to 100% for distances between 15 and 30meters 362, 3) emitter and detector experience 100% overlap fordistances between 30 and 45 meters 364, 4) emitter and detector overlapdecreases linearly from 100% to 0% for distances between 45 and 60meters 366.

In embodiments, the determination of distance for each pixel (m,n) for anon-overlapping range gating optical configuration with a descriptor of3/100/1/2/0 is illustrated in Table 6 below.

TABLE 6 White Black Condition Test Distance Range Point Point DistanceI₀(m, n) > I₁(m, n)? 0 m < d(m, n) < 15 m I₀(m, n) I₂(m, n) Eq. 26 I₁(m,n) > I₀(m, n) & I₁(m, n) > I₂(m, n)? 15 m < d(m, n) < 30 m I₁(m, n) Eq.27 Eq. 28 I₂(m, n) > I₁(m, n) & I₁(m, n) > I₀(m, n)? 30 m < d(m, n) < 45m I₂(m, n) I₀(m, n) Eq. 29 I₂(m, n) > I₁(m, n) & I₁(m, n) = I₀(m, n)? 45m < d(m, n) < 60 m >I₂(m, n) I₀(m, n) Eq. 30 I₂(m, n) = I₁(m, n) = I₀(m,n)? d(m, n) > 60 m n/a I₀(m, n) Eq. 31

In embodiments, when I₀(m,n)>I₁(m,n) the object at pixel (m,n) is in therange 0 m<d(m,n)<15 m and I₁(m,n) determines the actual distanceaccording to:

d(m,n)={[(I ₁(m,n)−I ₂(m,n))/(I ₀(m,n)−I ₂(m,n))]*C*P _(emitter)}/2  Eq.26

Where I₀(m,n) is the sub-frame 0 intensity value and the white pointvalue for pixel (m,n)

-   -   I₁(m,n) is the sub-frame 1 intensity value for pixel (m,n)    -   I₂(m,n) is the sub-frame 2 intensity value and the black point        value for pixel (m,n)    -   C is a constant for the speed of light in a medium    -   P_(emitter) is the emitter clock period

In embodiments, when I₁(m,n)>I₀(m,n) & I₁(m,n)>I₂(m,n) the object atpixel (m,n) is in the range 15 m<d(m,n)<30 m and the black point valueis determined according to:

BP(m,n)=(I ₁(m,n)−I ₀(m,n))+(I ₁(m,n)−I ₂(m,n))  Eq. 27

Where I₀(m,n) is the sub-frame 0 intensity value for pixel (m,n)

-   -   I₁(m,n) is the sub-frame 1 intensity value and the white point        value for pixel (m,n)    -   I₂(m,n) is the sub-frame 2 intensity value for pixel (m,n)

In embodiments, when I₁(m,n)>I₀(m,n) & I₁(m,n)>I₂(m,n) the object atpixel (m,n) is in the range 15 m<d(m,n)<30 m and the actual distance iscomputed according to:

d(m,n)={[1+(I ₂(m,n)−BP(m,n))/(I ₁(m,n)−BP(m,n))]*C*P _(emitter)}/2  Eq.28

Where I₁(m,n) is the sub-frame 1 intensity value and the white pointvalue for pixel (m,n)

-   -   I₂(m,n) is the sub-frame 2 intensity value for pixel (m,n)    -   BP(m,n) is the black point value for pixel (m,n) from Eq. 26    -   C is a constant for the speed of light in a medium    -   P_(emitter) is the emitter clock period

In embodiments, when I₂(m,n)>I₁(m,n) & I₁(m,n)>I₀(m,n) the object atpixel (m,n) is in the range 30 m<d(m,n)<45 m and the actual distance iscomputed according to:

d(m,n)={[2+(I ₂(m,n)−I ₁(m,n))/(I ₂(m,n)−I ₀(m,n))]*C*P_(emitter)}/2  Eq. 29

Where I₀(m,n) is the sub-frame 0 intensity value and the black pointvalue for pixel (m,n)

-   -   I₁(m,n) is the sub-frame 1 intensity value for pixel (m,n)    -   I₂(m,n) is the sub-frame 2 intensity value and the white point        value for pixel (m,n)    -   C is a constant for the speed of light in a medium    -   P_(emitter) is the emitter clock period

In embodiments, when I₂(m,n)>I₁(m,n) & I₁(m,n)=I₀(m,n) the black pointvalue is determined as I₀(m,n) and the white point value is undermined.Without knowledge of the white point the distance to the object at pixel(m,n) is in the range 45 m<d(m,n)<60 m and is determined according to:

(3*C*P _(emitter))/2>d(m,n)>(3*C*P _(emitter))/2  Eq. 30

In embodiments, when I₂(m,n)=I₁(m,n)=I₀(m,n) the black point value isdetermined as I₀(m,n) and the white point value is undermined. Withoutknowledge of the white point the distance to the object at pixel (m,n)is in the range d(m,n)>60 m and is determined according to:

d(m,n)>(4*C*P _(emitter))/2  Eq. 31

Increasing the number of sub-frames in a non-overlapping range gatingconfiguration increases the number of ranges for which distances aredetermined. Increasing the period of the emitter clock increases therange of each range gating cycle. In embodiments, the maximum ranges forwhich pixel distances are determined for varying numbers of range gatingcycles at varying emitter clock periods is expressed as:

Range_(max)=(N _(RG) *C*P _(emitter))/2  Eq. 32

Where N_(RG) is the number of non-overlapping range gating sub-frames

-   -   C is a constant for the speed of light in a medium    -   P_(emitter) is the emitter clock period

In embodiments, with a speed of light expresses as 0.299792 m/nSec, themaximum ranges for combinations of sub-frame numbers and emitter clockperiods are shown in Table 7 below.

TABLE 7 Number of Range Gate Sub-frames Emitter Clock Period Max Range 350 nSec 22.5 m 4 50 nSec 30.0 m 5 50 nSec 37.5 m 3 100 nSec 45.0 m 4 100nSec 60.0 m 5 100 nSec 74.9 m 3 100 nSec 89.9 m 4 100 nSec 119.9 m  5100 nSec 149.9 m 

High-dynamic-range imaging (HDR) imaging is a technique used in imagingto reproduce a greater dynamic range of luminosity than what is possiblewith standard digital imaging techniques, such as many real-world scenescontaining very bright, direct sunlight to extreme shade. HDR is oftenachieved by capturing and then combining several different,narrower-range exposures of the same subject matter. Non-HDR camerastake images_with a limited exposure range, referred to aslow-dynamic-range (I-DR), resulting in the loss of detail in highlightsor shadows. HDR images typically require little or motion by a camera orby objects within a scene. Table 3 identifies an eXtended Dynamic Range(XDR) waveform with a minimum of three sub-frames. In embodiments,sub-frames are collected at three different exposures withphotodetectors that exhibit a linear response to an incident number ofphotons. Intensity levels for the three or more XDR sub-frames areexpressed as I₀, I₁ through I_(X-1) where the intensity values are theresponse to three or more exposure levels, typically measured in numberof microseconds.

In embodiments, the fill rate of an XDR cycle expresses how rapidly apixel's intensity increases to a unit increase in exposure time. For athree sub-frame XDR cycle, the fill rate for sub-frames one and two foreach pixel is expressed as:

FillRate₁₋₂ =[I ₂(E ₂)−I ₁(E ₁)]/E ₂ −E ₁  Eq. 33

Where I₂ is the intensity for sub-frame 2

-   -   E₂ is the exposure time that produced 12    -   I₁ is the intensity level for I₁    -   E₁ is the exposure time that produced I₁

For a three sub-frame XDR cycle, the fill rate for sub-frames zero andone for each pixel is expressed as:

FillRate₀₋₁ =[I ₁(E ₁)−I ₀(E ₀)]/E ₁ −E ₀  Eq. 34

Where I₁ is the intensity for sub-frame 1

-   -   E₁ is the exposure time that produced I₁    -   I₀ is the intensity level for I₀    -   E₀ is the exposure time that produced I₀

The XDR intensity level for each pixel for sub-frames one and two isexpressed as:

I _(XDR)(E _(XDR))=FillRate₁₋₂*(E _(XDR) −E ₂)  Eq. 35

Where E_(XDR) is the exposure level for which XDR is computed

-   -   E₂ is the sub-frame 2 exposure time

The XDR intensity level for each pixel for sub-frames zero and one isexpressed as:

I _(XDR)(E _(XDR))=FillRate₀₋₁*(E _(XDR) −E ₁)  Eq. 36

Where E_(XDR) is the exposure level for which XDR is computed

-   -   E₁ is the sub-frame 1 exposure time

For purposes of describing the various embodiments, the followingterminology and references may be used with respect to reflectivearticles or materials in accordance with one or more embodiments asdescribed.

“Lighting-invariant imaging” describes a multi-frame, composite imagingsystem whereby maximum pixel intensity values and minimum pixelintensity values are determined for successive frames that constitute acomposite image.

“Black Point” refers to a frame pixel intensity value or a frame ofpixels whereby there existed no active light source or a low level ofactive light projected onto a scene during the photodetector integrationtime. The term black point is equivalent to the minimum pixel intensityin a Lighting-invariant imaging system.

“White Point” refers to a frame pixel intensity value or a frame ofpixels whereby there existed an active light projected onto a sceneduring photodetector integration time, whereby the intensity of thelight or the duration of the on time was greater than the intensity orthe duration of the associated black point intensity or duration. Theterm white point is equivalent to the maximum pixel intensity in aLighting-invariant imaging system.

“Luminance” describes the amount of radiant flux emitted or reflected bya surface per unit projected area due to one or more ambient lightsources, and is expressed in Watts/m².

“Radiance” describes the amount of radiant flux emitted or reflected bya surface per unit projected area due to a directed light source, and isexpressed in Watts/m².

“Spherical Coordinate System” is a three-dimensional coordinate spaceused for description of locations relative to a known point on a vehicleor an imaging component. Spherical coordinates are specified as (ρ,θ,φ),where p specifies distance, θ specifies the vertical angle, and yspecifies the horizontal or azimuth angle.

“Photodetector Accumulation Cycle” refers to accumulation of charge by aphotodetector for an accumulation duration followed by the transfer ofaccumulated photodetector charge to a storage element.

“Multiple Accumulation” refers to a process whereby more than onephotodetector accumulation cycle is performed within a photodetectorsub-frame event. The amplitude of collected charge at a storage elementis the sum of the accumulated photodetector charges that are transferredto the storage element within a multiple accumulation cycle.

“Frame” describes the electrical data produced by an imaging elementlike a focal plane array whereby optical information is converted toelectrical information for a multi-pixel device or system. Frameinformation is post-processed in an imaging system to convert a singleframe to an image. Focal plane arrays typically specify a capture andtransfer rate by utilizing a term like frames per second.

“Sub-frame” describes the electrical data produced by an imaging elementlike a focal plane array whereby optical information is converted toelectrical information for a multi-pixel device or system. Sub-frameinformation is post-processed in an imaging system to convert multiplesub-frames to a composite image or multiple composite images.

A “sub-frame trapezoidal descriptor” defines the electro-opticalparameters of a sub-frame composite imaging cycle whereby the timingrelationship of an emitter and a detector is different for subsequentsub-frames within an imaging duration, with the descriptor defined by aformat:

<# of sub-frames>/ <emitter clock period (in nSec)>/ <# of emitter clockperiods for emitter pulses>/ <# of emitter clock periods for detectorintegration>/ <# of emitter clock periods between end of integration andstart of emitter pulse for sub-  frame 0> <sub-frame period duration,defined as the elapsed time from the start of a sub-frame to  the startof a subsequent sub-frame within an imaging cycle (in μSec)>.

A “sub-frame WP/BP descriptor” defines the electro-optical parameters ofa sub-frame composite imaging cycle whereby white point sub-frames andblack point sub-frames are produced alternately throughout the imagingwindow, with the descriptor defined by a format:

<# of sub-frames>/ <sub-frame period duration, defined as the elapsedtime from the start of a sub-frame to  the start of a subsequentsub-frame within an imaging cycle (in μSec)>.

“Range Gating” describes an active sensor imaging technique that allowsfor the imaging of an object within a distance band from a sensor. Inrange-gated imaging, a pulsed light source is used to illuminate a scenewhile reflected light is detected by a sensor with a short exposure timeor a short integration time referred to as a gate. The gate is delayedso imaging occurs at a particular range from the sensor.

“Non-overlapping range gating” describes the use of multiple range gatesin a sub-frame, composite imaging system whereby the maximum distance ofa range gate equates to the minimum distance of a subsequent range gate.Non-overlapping range-gating composite imagery requires a minimum of twosub-frames per composite image.

A “sub-frame non-overlapping range gating descriptor” defines theelectro-optical parameters of a sub-frame composite imaging cyclewhereby the timing relationship of an emitter and a detector isdifferent for subsequent sub-frames within an imaging duration, andwhereby there exists no overlap between the range at which the maximumintensity of one sub-frame range overlaps with the maximum intensity ofa previous or subsequent sub-frame within a composite image, with thedescriptor defined by a format:

<# of sub-frames>/ <emitter clock period (in nSec)>/ <# of emitter clockperiods for emitter pulses>/ <# of emitter clock periods for detectorintegration>/ <# of emitter clock periods between start of integrationand start of emitter pulse for sub-  frame 0>.

Persons of ordinary skill in the relevant arts will recognize thatembodiments may comprise fewer features than illustrated in anyindividual embodiment described above. The embodiments described hereinare not meant to be an exhaustive presentation of the ways in which thevarious features of the embodiments may be combined. Accordingly, theembodiments are not mutually exclusive combinations of features; rather,embodiments can comprise a combination of different individual featuresselected from different individual embodiments, as understood by personsof ordinary skill in the art. Moreover, elements described with respectto one embodiment can be implemented in other embodiments even when notdescribed in such embodiments unless otherwise noted. Although adependent claim may refer in the claims to a specific combination withone or more other claims, other embodiments can also include acombination of the dependent claim with the subject matter of each otherdependent claim or a combination of one or more features with otherdependent or independent claims. Such combinations are proposed hereinunless it is stated that a specific combination is not intended.Furthermore, it is intended also to include features of a claim in anyother independent claim even if this claim is not directly madedependent to the independent claim.

Any incorporation by reference of documents above is limited such thatno subject matter is incorporated that is contrary to the explicitdisclosure herein. Any incorporation by reference of documents above isfurther limited such that no claims included in the documents areincorporated by reference herein. Any incorporation by reference ofdocuments above is yet further limited such that any definitionsprovided in the documents are not incorporated by reference hereinunless expressly included herein.

For purposes of interpreting the claims, it is expressly intended thatthe provisions of Section 112, sixth paragraph of 35 U.S.C. are not tobe invoked unless the specific terms “means for” or “step for” arerecited in a claim.

1-11. (canceled)
 12. An imaging system configured to generate acomposite image two-axis motion map of a scene at a pixel level, theimaging system comprising: at least one emitter configured to emit anactive light pulse toward the scene; an array of detectors configured toreceive light that includes some of the active light pulse reflectedfrom the scene for a field of view that includes at least a portion ofthe scene, each detector in the array of detectors configured to producea linearized intensity response to a number of incident photons oflight; control circuitry operably coupled to the at least one emitterand the array of detectors and configured to store a set of at leastthree successive sub-frames of intensity data as sub-frame pixels in oneor more sub-frame buffers, wherein each sub-frame pixel has a timingrelationship of an emitter/detector cycle for that sub-frame pixel andeach set of sub-frame pixels associated with a unique one of an array ofpixels based on a row and a column corresponding to the array ofdetectors; and a processing system operably coupled to the controlcircuitry and the one or more sub-frame buffers to generate thecomposite image two-axis motion map of the scene, the processing systemconfigured to: analyze at least three successive sub-frame pixels todetermine for each pixel in the array of pixels a black point due toambient light in the scene and a white point due to the active lightpulse reflected from the scene for at least the first sub-frame pixeland the last sub-frame pixel for the set of sub-frame pixels for thatpixel; generate a horizontal axis motion value for each pixel relativeto a row in the pixel array based on a high-intensity rate of changebetween that pixel and at least one neighbor pixel in the row; andgenerate a vertical axis motion value for each pixel relative to acolumn in the pixel array based on a high-intensity rate of changebetween that pixel and at least one neighbor pixel in the column. 13.The imaging system of claim 12 wherein the processing system determinesthe high-intensity rate of change by evaluating a sub-frame pixel atwhich a slope of the white point crosses over a trailing edge of theblack point for a neighbor pixel in a pixel triplet for a given row orcolumn of the pixel array that has a slope that is non-zero.
 14. Theimaging system of claim 12 wherein a duration of a capture cycle isconstant for the at least three successive sub-frame pixels, and anintensity and a duration of the active light pulse emitted during thecapture cycle is the same for the first sub-frame pixel and the lastsub-frame pixel, but the intensity and the duration of the active lightpulse is different for at least one sub-frame pixel between the firstsub-frame pixel and the last sub-frame pixel.
 15. The imaging system ofclaim 12 wherein a duration of a capture is the same for the firstsub-frame pixel and the last sub-frame pixel, but the duration of acapture is shorter for at least one sub-frame pixel between the firstsub-frame pixel and the last sub-frame pixel.
 16. The imaging system ofclaim 12 wherein the imaging system is mounted in a vehicle capable ofmoving at speeds of more than 50 km/hour and all of the three or moresub-frame pixels for each pixel are stored within an imaging window lessthan 250 μSec.
 17. The imaging system of claim 12 wherein the imagingsystem is mounted in a handheld device and the three or more sub-framepixels for each pixel are stored within an imaging window of less than2500 μSec.
 18. The imaging system of claim 12 wherein the processingsystem, the array of detectors, the control circuitry, and theprocessing system are integrated on a single chip.
 19. The imagingsystem of claim 12 wherein the processing system, the array ofdetectors, and the control circuitry are integrated on a single chip andthe processing system is external to the single chip.
 20. The imagingsystem of claim 12 wherein the active light pulse in a givenemitter/detector cycle for a given pixel comprises: a number of pulsesselected from the set consisting of a single pulse per pixel, a sequenceof multiple pulses per pixel, a single pulse per sub-pixel, or multiplepulses per sub-pixel, and a frequency selected from the set consistingof a single frequency range or multiple frequency ranges. 21-35.(canceled)
 36. The imaging system of claim 12 wherein the array ofdetectors is configured to accumulate light based on a singleaccumulation for the timing relationship of the emitter/detector cyclethat is unique for each sub-pixel.
 37. The imaging system of claim 12wherein the array of detectors is configured to accumulate light basedon a plurality of accumulations for the timing relationship of theemitter/detector cycle that is the same for each sub-pixel.
 38. Animaging system configured to generate a composite image three-axismotion map of a scene at a pixel level, the imaging system comprising:at least one emitter configured to emit an active light pulse toward thescene; an array of detectors configured to receive light that includessome of the active light pulse reflected from the scene for a field ofview that includes at least a portion of the scene, each detector in thearray of detectors configured to produce a linearized intensity responseto a number of incident photons of light; control circuitry operablycoupled to the at least one emitter and the array of detectors andconfigured to store at least six successive sub-frames of intensity dataas sub-frame pixels in one or more sub-frame buffers, wherein eachsub-frame pixel has a timing relationship of an emitter/detector cyclefor that sub-frame pixel and each set of sub-frame pixels associatedwith a unique one of an array of pixels based on a row and a columncorresponding to the array of detectors with at least three sub-framesdedicated to two-axis motion, and at least three sub-frames dedicated todepth map computation; and a processing system operably coupled to thecontrol circuitry and the one or more sub-frame buffers to generatethree-axis motion for pixels of the scene, the processing systemconfigured to: analyze the at least three successive sub-frame pixels todetermine for a pixel associated with the sub-frame pixels a blackpoint, a white point, and the sub-frame pixel at which the white pointoccurs; determine a distance for each pixel based on the sub-frame pixelat which the white point occurs; analyze at least three successivesub-frame pixels to determine for each pixel in the array of pixels ablack point due to ambient light in the scene and a white point due tothe active light pulse reflected from the scene for at least the firstsub-frame pixel and the last sub-frame pixel for the set of sub-framepixels for that pixel; generate a horizontal axis motion value for eachpixel relative to a row in the pixel array based on a high-intensityrate of change between that pixel and at least one neighbor pixel in therow; and generate a vertical axis motion value for each pixel relativeto a column in the pixel array based on a high-intensity rate of changebetween that pixel and at least one neighbor pixel in the column. 39.The imaging system of claim 38 wherein the processing system determinesthe high-intensity rate of change by evaluating a sub-frame pixel atwhich a slope of the white point crosses over a trailing edge of theblack point for a neighbor pixel in a pixel triplet for a given row orcolumn of the pixel array that has a slope that is non-zero.
 40. Theimaging system of claim 38 wherein a duration of a capture cycle isconstant for the at least three successive sub-frame pixels, and anintensity and a duration of the active light pulse emitted during thecapture cycle is the same for the first sub-frame pixel and the lastsub-frame pixel, but the intensity and the duration of the active lightpulse is different for at least one sub-frame pixel between the firstsub-frame pixel and the last sub-frame pixel.
 41. The imaging system ofclaim 38 wherein a duration of a capture is the same for the firstsub-frame pixel and the last sub-frame pixel, but the duration of acapture is shorter for at least one sub-frame pixel between the firstsub-frame pixel and the last sub-frame pixel.
 42. The imaging system ofclaim 38 wherein the distance represented by each sub-frame pixel isdefined by an overlap in a duration of the timing relationship of theemitter/detector cycle for that sub-frame pixel.
 43. The imaging systemof claim 38 wherein the processing system, the array of detectors, thecontrol circuitry, and the processing system are integrated on a singlechip.
 44. The imaging system of claim 38 wherein the active light pulsein a given emitter/detector cycle for a given pixel comprises: a numberof pulses selected from the set consisting of a single pulse per pixel,a sequence of multiple pulses per pixel, a single pulse per sub-pixel,or multiple pulses per sub-pixel, and a frequency selected from the setconsisting of a single frequency range or multiple frequency ranges. 45.The imaging system of claim 38 wherein the array of detectors isconfigured to accumulate light based on a single accumulation for thetiming relationship of the emitter/detector cycle that is unique foreach sub-pixel.
 46. The imaging system of claim 38 wherein the array ofdetectors is configured to accumulate light based on a plurality ofaccumulations for the timing relationship of the emitter/detector cyclethat is the same for each sub-pixel.